Node.js Deployment & Debugging Runbook AITC Documentation Hub
Documentation Hub
AITC Production Standard

Node.js Web Application Deployment & Debugging Runbook

Reusable guidance for CAFNR/AITC applications hosted on IIS using ARR reverse proxy, PM2, Node.js/Express, secure sessions, and ExcelJS-based data storage. This runbook captures the deployment and debugging lessons from CRNeeds so future applications can avoid the same issues.

1. Standard Architecture

Use the architecture already proven on the AITC server. Do not assume IIS-hosted Node applications should use iisnode.

Internet ↓ HTTPS ↓ IIS ↓ ARR / URL Rewrite ↓ 127.0.0.1 : Node Port ↓ Node.js / Express ↓ PM2
AITC server standard: IIS/ARR → localhost Node → PM2.

2. IIS Application Configuration

Reference deployment:

IIS application:
/AITC_Website/CAFNR/CRNeeds

Physical path:
C:\inetpub\wwwroot\AITC_Website\CAFNR\CRNeeds

Public URL:
https://aitc.pvamu.edu/CAFNR/CRNeeds

The public URL may not include every IIS parent application segment because higher-level rewrite behavior may already exist.

3. ARR Reverse Proxy Configuration

Use an application-relative URL Rewrite rule:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <rule name="ReverseProxyToNode" stopProcessing="true">
          <match url="^(.*)$" />
          <action
            type="Rewrite"
            url="http://127.0.0.1:3009/{R:1}"
            appendQueryString="true" />
        </rule>
      </rules>
    </rewrite>
    <httpErrors existingResponse="PassThrough" />
  </system.webServer>
</configuration>
Avoid: forwarding {UNENCODED_URL} from a nested IIS application when the Node routes are root-relative.

Correct request flow

Browser:
/CAFNR/CRNeeds/assessment

IIS application-relative request:
assessment

ARR forwards:
http://127.0.0.1:3009/assessment

Express receives:
/assessment

4. BASE_PATH vs Node Routes

Keep Express routes root-relative:

router.get('/assessment', ...)
router.get('/login', ...)
router.get('/dashboard', ...)

Use the public prefix only for generated links and redirects:

BASE_PATH=/CAFNR/CRNeeds
Browser URLNode route
/CAFNR/CRNeeds/dashboard/dashboard
/CAFNR/CRNeeds/login/login

5. PM2 Process Management

pm2 start server.js --name crneeds
pm2 list
pm2 logs crneeds --lines 30 --nostream

Do not save the PM2 process list simply because the process reports online.

PM2 online → Public page → Login → Database write → Dashboard → Logs clean → pm2 save
pm2 save

6. Windows Service Identity & NTFS Permissions

A successful manual test by an administrator does not prove the PM2 service account can write to the same folders.

CRNeeds ran under:

NT AUTHORITY\LOCAL SERVICE
PathRecommended permission
Application treeRead / Execute
data\Modify
backups\Modify
sessions\Modify

Typical failures:

EPERM copyfile
EPERM open
EPERM rename

7. Keep the Live Excel Database Local

C:\inetpub\wwwroot\AITC_Website\CAFNR\CRNeeds\data\CRNeeds.xlsx
Do not use OneDrive or another sync folder as the live application database. Sync clients can introduce locks, conflicts, and replacement behavior.

8. Do Not Open the Live Workbook in Excel

Desktop Excel may lock the XLSX file and prevent application writes.

EPERM
EBUSY
rename failure
write failure

Provide an application export instead of having users open the production workbook.

9. Serialize Excel Writes

A single XLSX file is not a multi-user database. Multiple simultaneous writes must be serialized.

this.queue = Promise.resolve();

_runExclusive(task) {
    const next = this.queue.then(task, task);
    this.queue = next.catch(() => {});
    return next;
}

Without serialization:

User A loads workbook
User B loads workbook
User A writes response
User B writes older workbook
User A's response disappears

10. Critical ExcelJS Rule: Do Not Rely on Column Keys After Reload

ExcelJS column key metadata can be used while creating a workbook but should not be relied upon after the XLSX file is saved and reopened.

Unsafe persistent write

ws.addRow({
    responseId: responseId,
    submittedAt: submittedAt
});

Production-safe write

ws.addRow([
    responseId,
    submittedAt,
    ipHash,
    userAgent
]);
Use positional arrays for persistent new rows in Responses, Users, LeadershipRatings, AuditLog, and Metadata.

11. Use Atomic Workbook Writes

1. Load production workbook
2. Modify workbook in memory
3. Write complete temporary XLSX
4. Rename temporary file to production path
5. Remove old file after successful replacement

Example:

CRNeeds.xlsx.tmp-1234-1791421330000.xlsx

12. Daily Backups

backups\CRNeeds-2026-10-08.xlsx

Create a recovery copy before production writes. One automatic backup per day is generally sufficient for this application pattern.

13. Use HTTP Status Codes to Narrow the Problem

StatusMeaning during form testing
200Page/request returned successfully
302POST accepted and application redirected
400Application validation rejected the submission
500Backend/application failure
A successful 302 followed by missing data strongly suggests a persistence-layer issue rather than IIS routing.

14. Form Validation Can Look Like a Database Problem

A survey may intentionally return HTTP 400 without creating a PM2 exception.

Question 31:
Exactly three investment priorities.

Question 32:
Must be one of the Question 31 selections.

Required matrix:
Every required row must be completed.

15. Secure Express Sessions Behind IIS/ARR

const app = express();

app.set('trust proxy', 1);
app.use(session({
    proxy: true,

    store: new FileStore({
        path: path.resolve('./sessions'),
        ttl: 60 * 60 * 8,
        retries: 0
    }),

    name: 'crneeds.sid',
    secret: process.env.SESSION_SECRET,

    resave: false,
    saveUninitialized: false,
    rolling: true,

    cookie: {
        httpOnly: true,
        sameSite: 'lax',
        secure:
            String(process.env.COOKIE_SECURE || 'false')
                .toLowerCase() === 'true',
        maxAge: 1000 * 60 * 60 * 8,
        path: '/'
    }
}));

Production:

COOKIE_SECURE=true

17. Bind Node Only to Localhost

const port = Number(process.env.PORT || 3088);

store.init()
  .then(() => {
    app.listen(port, '127.0.0.1', () => {
      console.log(`Application listening on 127.0.0.1:${port}`);
    });
  })
  .catch(err => {
    console.error('Unable to initialize backend:', err);
    process.exit(1);
  });

Verify:

Get-NetTCPConnection -LocalPort 3009 -State Listen |
Select-Object LocalAddress,LocalPort,OwningProcess

Expected:

127.0.0.1    3009

18. ARR Forwarded IP and Express Rate Limit

ARR may present an address such as:

10.10.39.227:61413

Express Rate Limit may reject it with:

ERR_ERL_INVALID_IP_ADDRESS

Normalize the client address before using it as a rate-limit key:

function rateLimitKey(req) {
    let ip = String(
        req.ip ||
        req.socket?.remoteAddress ||
        'unknown'
    ).trim();

    if (/^\d{1,3}(?:\.\d{1,3}){3}:\d+$/.test(ip)) {
        ip = ip.replace(/:\d+$/, '');
    }

    if (ip.startsWith('::ffff:')) {
        ip = ip.substring(7);
    }

    const match = ip.match(/^\[([^\]]+)\](?::\d+)?$/);
    if (match) ip = match[1];

    return ip;
}

19. Production .env Pattern

NODE_ENV=production

PORT=3009

BASE_PATH=/CAFNR/CRNeeds

SESSION_SECRET=<strong-random-secret>

SUPERADMIN_USERNAME=<admin-account>

SUPERADMIN_PASSWORD=<strong-password>

COOKIE_SECURE=true

DATA_FILE=./data/CRNeeds.xlsx

BACKUP_DIR=./backups

APP_TITLE=CAFNR Research Living Laboratory Needs Assessment
Never place real passwords or session secrets in documentation, source control, email, Teams messages, or public repositories.

20. Recommended Application Folder Structure

AppRoot\
│
├── server.js
├── package.json
├── .env
├── web.config
│
├── public\
├── views\
├── src\
│
├── data\
├── backups\
└── sessions\

21. Standard Deployment Sequence

22. Production Smoke Test

Public

Storage

Authentication

Administration

DSS

Infrastructure

23. Useful Diagnostic Commands

PM2

pm2 list

pm2 logs crneeds --lines 50 --nostream

pm2 restart crneeds

pm2 flush

pm2 save

Verify Node listener

Get-NetTCPConnection -LocalPort 3009 -State Listen |
Select-Object LocalAddress,LocalPort,OwningProcess

Workbook timestamp

Get-Item ".\data\CRNeeds.xlsx" |
Select-Object Length,LastWriteTime

Count response rows

node -e "const ExcelJS=require('exceljs');(async()=>{const wb=new ExcelJS.Workbook();await wb.xlsx.readFile('./data/CRNeeds.xlsx');const ws=wb.getWorksheet('Responses');console.log('Rows including header:',ws.rowCount);})().catch(console.error)"

JavaScript syntax check

node --check ".\server.js"

node --check ".\src\services\excelStore.js"

Recent IIS log

$log = Get-ChildItem `
"C:\inetpub\logs\LogFiles\W3SVC1" `
-Filter *.log |
Sort-Object LastWriteTime -Descending |
Select-Object -First 1

Select-String `
  -Path $log.FullName `
  -Pattern "CRNeeds" |
Select-Object -Last 20

24. Symptom → Likely Cause

SymptomLikely area
IIS 500 before application startupIIS / web.config
Node works directly but public URL failsARR / URL Rewrite
Homepage works but nested route returns 404Incorrect prefix forwarding
EPERM writing XLSXPM2 identity / NTFS permissions / file lock
PM2 continually restartsPM2 error log / startup failure
Login redirects to Access DeniedSession cookie / proxy configuration
Login works only with COOKIE_SECURE=falseHTTPS proxy awareness
Form POST returns 400Application validation
Form POST returns 302 but no data appearsPersistence layer
XLSX timestamp changes but no row appearsExcelJS object/key addRow issue
Dashboard shows zeroInspect Responses worksheet before dashboard code
ERR_ERL_INVALID_IP_ADDRESSARR forwarded IP contains source port
Node reachable directly from networkBind Node to 127.0.0.1
Writes intermittently failWorkbook open in Excel / permissions
Responses disappear under concurrent useWrites are not serialized

25. Five Mandatory Lessons

1. Copy a known-good server architecture first.
Inspect an existing functioning AITC application before designing a new deployment configuration.
2. Use IIS/ARR + PM2 as the standard architecture.
Do not introduce iisnode unless there is a deliberate infrastructure decision to change the server standard.
3. Identify the actual PM2 service account.
A successful manual test by an administrator does not establish that the production process can access the same resources.
4. Use positional ExcelJS writes.
Do not depend on worksheet column keys after reopening an XLSX workbook.
5. Test real persistence before declaring success.
A 200 response, 302 redirect, or PM2 online state does not prove that the application successfully saved or retrieved data.