Hosting / PaaS / Northflank

Northflank

SEV 2 · BLEEDING · 17h 10m

Since Jun 2025 · 25 incidents · status.northflank.com · checked just now

User reportsVisitors who said it's down for them, per hour over 24 hours. Not from the status page. Never counted in grades.
…
GradeCLast 90 days
BPRODASHIP
Happening now
Oct 30, 2026ongoingSEV 2 · SECURITY
Security incidentNorthflank detected a threat alert and found attackers may have decrypted customer environment variables. Users were told to rotate all secrets and disable public addon access. Users reported unauthorized use of AI API keys stored on the platform. Secret rotation was enabled for Postgres and MongoDB, and audit logs were enabled for all customers. Investigation is ongoing and the incident remains open.AI-graded · some customers · Builds +6
Why this grade
  1. RULES · SEV 1
    Sev 1 because Networking had a major outage, plus 600 points for a security compromise
  2. CLAUDE · SEV 2 · SUBSET · MEDIUM CONFIDENCE · SECURITY
    A security incident exposed customer environment variables and AI keys were used without authorization; scope across customers is not stated.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
6 updates
  1. 13:40 UTC · investigating
    Rotate environment variables immediately. 11:43 UTC: Wiz threat alert 11:44 UTC: Northflank engineers review the alert 12:02 UTC: Initial network policy patch 12:50 UTC: Additional patching deployed 13:00 UTC: Wiz CIRT support requested 13:05 UTC: Additional patching deployed 13:05 UTC: Additional patching deployed
  2. 13:58 UTC · identified
    Assume all environment variables have been decrypted. Disable/rotate all keys.
  3. 15:12 UTC · identified
    Secret rotation on Postgres and MongoDB has been enabled for all users. We recommend disabling public expose on addons if you have enabled that. All users should assume their environment variables are decrypted and begin rotation immediately. Secrets on workloads Secret groups Global secrets Multiple users are confirming that their AI keys have been used.
  4. 19:28 UTC · identified
    Our investigation is ongoing. We have no further confirmed findings to share at this time. Our guidance remains unchanged: assume all environment variables have been compromised and rotate all secrets and credentials immediately, including those in workloads, secret groups and global secrets. Disable public access to addons where enabled. We will share further updates as more information is confirmed.
  5. 23:38 UTC · identified
    Users have reported unauthorised use of AI API keys supplied to the Northflank platform. If you have not already rotated your user-provided secrets stored in Northflank, please do so immediately. Revoke existing credentials with their issuing providers, generate replacements, and update the affected secrets and workloads in Northflank. Review your AI provider usage and billing logs for unexpected activity, and report any suspicious usage to Northflank support and the relevant provider. These reports may be consistent with the broader pattern of attacks targeting AI credentials. Similar IP addresses are contained in this article [Wiz’s “Black October” report](https://www.wiz.io/blog/black-october-for-ai-developer-platforms).
  6. 01:04 UTC · identified
    We have enabled audit logs for all customers, including free users.
7 components affected7 major outage
All 7
  • Addonsmajor outage
  • Buildsmajor outage
  • Certificatesmajor outage
  • Jobsmajor outage
  • Logs and Metricsmajor outage
  • Networkingmajor outage
  • Servicesmajor outage

Overview

Admitted uptimeCustomer time not hit, by the provider's own account. Regional and partial incidents count in part.
99.874%Grade C
IncidentsEverything posted on the status page except scheduled maintenance.
50 Sev 1
Fully downMost customers down, by the provider's own account.
0m
Median fixMedian time to resolve. Half of incidents were fixed faster.
2h 25m
LongestLongest Sev 1–2 incident, start to resolved.
17h 10m
Since lastTime since the last counted incident was resolved.
ongoing

By area

BProduction
2INCIDENTS
18h 11mAFFECTED
Your users are hit: apps, APIs, databases, inference
Worst, 90d
SEV 2 · SECURITYOct 30, 2026 · ongoing · some customersSecurity incident
SEV 2Aug 5, 2026 · 1h 01m · regionalWorkloads Failing to Start Across Multiple Regions
Most hit
Addons2
Jobs2
Services2
Builds1
Worse. 6 in the last 6 months, 3 in the 6 before.
AShipping
5INCIDENTS
28h 13mAFFECTED
You can't deploy, build or manage: CI, CLI, dashboard, auth, billing
Worst, 90d
SEV 3 · SECURITYOct 30, 2026 · ongoing · some customersSecurity incident
SEV 3Sep 25, 2026 · 46mDegraded SSO login via WorkOS
SEV 3Oct 20, 2026 · 6h 51m · regionalPartial Control Plane Degredation
Most hit
Addons3
Logs and Metrics2
Jobs2
Services2
Worse. 10 in the last 6 months, 3 in the 6 before.
CleanSev 3–4Sev 1–2No data

What it costs

One small production web app, per month512 MB app on all month, 1M requests, 100 GB out, commercial use, one seat. Cheapest plan that runs it.
Pay as you go, nf-compute-204 of 9 cheaper · Heroku: cheaper, better grade
$11.40/mo
nf-compute-20 (0.2 vCPU, 512 MB)$5.40
100 GB egress at $0.06/GB$6

The free Sandbox tier caps services and databases, so a production app goes on pay as you go. northflank.com · checked Oct 31, 2026 · Wrong price?

Trend, last 12 months

Worse. 13 in the last 6 months, 6 in the 6 before.

Severity mix

Sev 1 · Lights outProd site, API or database down, or data loss0
Sev 2 · BleedingProd degraded or partly down2
Sev 3 · Can't shipCan't deploy, build or manage it3
Sev 4 · PapercutDocs, status page, slow dashboard0
1 security incident

When it breaks

Start times, last 12 months, UTC.

What breaks

Addons11 incidents
Services9 incidents
Jobs6 incidents
Networking4 incidents
Northflank App4 incidents
Certificates3 incidents

Where it breaks

Last 12 monthsPlaces named on the status page: cloud regions, cities, countries, data centres. An incident naming only a broad area (Europe, APAC) is counted under the map, not pinned.
2 places · 14 not pinned
London: 3 incidents · NorthflankNetherlands: 2 incidents · Northflank
Sev 1–2 time by place, 12 monthsTime with a Sev 1 or 2 open that names the place, any provider. Overlaps count once.
London7h 01m

Communication

Updates eachStatus page updates per incident, on average.
3.2Last 12 months
Update gapMedian time between status page updates during an incident.
52mLast 12 months
PostmortemsSev 1–2 incidents with a published write-up.
0 of 9Sev 1–2

Incident history

Oct 30, 2026ongoingSEV 2 · SECURITY
Security incidentNorthflank detected a threat alert and found attackers may have decrypted customer environment variables. Users were told to rotate all secrets and disable public addon access. Users reported unauthorized use of AI API keys stored on the platform. Secret rotation was enabled for Postgres and MongoDB, and audit logs were enabled for all customers. Investigation is ongoing and the incident remains open.AI-graded · some customers · Builds +6
Why this grade
  1. RULES · SEV 1
    Sev 1 because Networking had a major outage, plus 600 points for a security compromise
  2. CLAUDE · SEV 2 · SUBSET · MEDIUM CONFIDENCE · SECURITY
    A security incident exposed customer environment variables and AI keys were used without authorization; scope across customers is not stated.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
6 updates
  1. 13:40 UTC · investigating
    Rotate environment variables immediately. 11:43 UTC: Wiz threat alert 11:44 UTC: Northflank engineers review the alert 12:02 UTC: Initial network policy patch 12:50 UTC: Additional patching deployed 13:00 UTC: Wiz CIRT support requested 13:05 UTC: Additional patching deployed 13:05 UTC: Additional patching deployed
  2. 13:58 UTC · identified
    Assume all environment variables have been decrypted. Disable/rotate all keys.
  3. 15:12 UTC · identified
    Secret rotation on Postgres and MongoDB has been enabled for all users. We recommend disabling public expose on addons if you have enabled that. All users should assume their environment variables are decrypted and begin rotation immediately. Secrets on workloads Secret groups Global secrets Multiple users are confirming that their AI keys have been used.
  4. 19:28 UTC · identified
    Our investigation is ongoing. We have no further confirmed findings to share at this time. Our guidance remains unchanged: assume all environment variables have been compromised and rotate all secrets and credentials immediately, including those in workloads, secret groups and global secrets. Disable public access to addons where enabled. We will share further updates as more information is confirmed.
  5. 23:38 UTC · identified
    Users have reported unauthorised use of AI API keys supplied to the Northflank platform. If you have not already rotated your user-provided secrets stored in Northflank, please do so immediately. Revoke existing credentials with their issuing providers, generate replacements, and update the affected secrets and workloads in Northflank. Review your AI provider usage and billing logs for unexpected activity, and report any suspicious usage to Northflank support and the relevant provider. These reports may be consistent with the broader pattern of attacks targeting AI credentials. Similar IP addresses are contained in this article [Wiz’s “Black October” report](https://www.wiz.io/blog/black-october-for-ai-developer-platforms).
  6. 01:04 UTC · identified
    We have enabled audit logs for all customers, including free users.
Oct 20, 20266h 51mSEV 3
Partial Control Plane DegredationA GCP europe-west2 zone had VM-to-VM communication problems. Logging and metrics were moved to another zone, then login and control plane services were hit. Deployed workloads were unaffected. Services returned to normal at 09:53 UTC and the incident was resolved at 13:34 UTC.AI-graded · regional · Logs and Metrics +1
Why this grade
  1. RULES · SEV 3
    Sev 3 because Logs and Metrics had degraded performance
  2. CLAUDE · SEV 3 · REGIONAL · MEDIUM CONFIDENCE
    Control plane and login degraded in one GCP europe-west2 zone for about 7 hours; existing customer workloads kept running.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
5 updates
  1. 06:44 UTC · identified
    Availability zone b in GCP europe-west2 is experiencing issues with VM to VM communication impacting parts of the platform. We are working on mitigating the impact. The majority of users and system components are unaffeced.
  2. 07:46 UTC · monitoring
    We have moved the affected parts of the logging and metric components to a different zone. We will continue to monitor the situation. A public Google status page is now available: <https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPyzj>
  3. 09:40 UTC · investigating
    The incident has expanded to our core control plane services, affecting login and control plane components. We are working to mitigate the impact. Existing user workloads deployed are unaffected.
  4. 09:53 UTC · monitoring
    Services are operating normally, and we are monitoring the situation.
  5. 13:34 UTC · resolved
    This incident has been resolved.
Sep 25, 202646mSEV 3
Degraded SSO login via WorkOSSSO login via WorkOS was degraded for about 46 minutes; the text does not say how many users were affected.AI-graded · Northflank App
Why this grade
  1. RULES · SEV 4
    Sev 4 because Northflank App had degraded performance
  2. CLAUDE · SEV 3 · UNCLEAR · LOW CONFIDENCE
    SSO login via WorkOS was degraded for about 46 minutes; the text does not say how many users were affected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 15:15 UTC · investigating
    We are currently investigating this incident.
  2. 16:01 UTC · resolved
    This incident has been resolved.
Sep 4, 20262h 25mSEV 3
Addon expose/unexpose temporarily disabled in US - CentralAddon expose and unexpose was disabled in US-Central for about 2.5 hours during a cloud provider incident; other regions unaffected.AI-graded · regional · Addons
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 3 · REGIONAL · MEDIUM CONFIDENCE
    Addon expose and unexpose was disabled in US-Central for about 2.5 hours during a cloud provider incident; other regions unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 18:27 UTC · monitoring
    We have temporarily disabled addon expose/unexpose in our US - Central region. This is due to an ongoing incident with our cloud service provider. We will re-enable this feature once we confirm that it is safe to do so.
  2. 20:52 UTC · resolved
    This incident has been resolved.
Aug 5, 20261h 01mSEV 2
Workloads Failing to Start Across Multiple RegionsWorkloads failed to start across multiple regions for about an hour; the text gives no share of workloads hit.AI-graded · regional · Addons +2
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE
    Workloads failed to start across multiple regions for about an hour; the text gives no share of workloads hit.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 13:27 UTC · investigating
    We are seeing partial start up issues for workloads across multiple regions. We are currently investigating this incident.
  2. 13:42 UTC · monitoring
    We have identified the issue and applied a mitigation. Previously stuck workloads are now starting successfully across affected regions. We are monitoring recovery and will confirm once all workloads are fully operational.
  3. 14:27 UTC · resolved
    All affected workloads are now fully operational.
Jul 22, 20261h 22mSEV 2
Investigating node stability issues in the US - East regionNodes in Northflank's US-East region failed for about 80 minutes, hitting services, jobs and add-ons there, caused by a Google Cloud outage.AI-graded · regional · Addons +2
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE
    Nodes in Northflank's US-East region failed for about 80 minutes, hitting services, jobs and add-ons there, caused by a Google Cloud outage.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 12:17 UTC · investigating
    We have noticed increased failure rates of nodes in the US - East region and are investigating the issue.
  2. 13:11 UTC · monitoring
    We are seeing recovery across compute and storage. We are working to get the remaining workloads back online.
  3. 13:39 UTC · resolved
    We have recovered affected workloads in the region and there are no more on-going node failures. The root cause was an outage on the Google Cloud Platform in us-east4 which hosts Northflank's US - East region.
Jul 21, 202632mSEV 2 · SECURITY
Active DDoS Attack in US - CentralA DDoS attack hit public ingress into the Northflank cluster in the US-Central region, first from 11:09 to 11:41 UTC and again from 13:23 to 13:58 UTC on 21 July 2026. Internal traffic was not affected. Mitigations were applied and the attack stopped each time.AI-graded · regional · Networking
Why this grade
  1. RULES · SEV 2
    Sev 2 because Networking had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE · SECURITY
    A DDoS attack disrupted public ingress to customer workloads in US-Central twice for under an hour each; internal traffic was unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
6 updates
  1. 11:09 UTC · identified
    We are currently experiencing a DDoS attack in the US - Central region. This is affecting public ingress into the cluster. Internal traffic is not affected. The team is working on additional mitigations.
  2. 11:20 UTC · monitoring
    We have implemented mitigations are monitoring the situation.
  3. 11:41 UTC · resolved
    The DDoS attack has stopped, we will continue to monitor the affected region.
  4. 13:23 UTC · identified
    We are currently experiencing a DDoS attack in the US - Central region. This is affecting public ingress into the cluster. Internal traffic is not affected. The team is working on additional mitigations.
  5. 13:43 UTC · monitoring
    We have implemented mitigations are monitoring the situation.
  6. 13:58 UTC · resolved
    This incident has been resolved. We will continue to monitor the affected region.
Jun 18, 20264h 40mSEV 3
Let's Encrypt Certificate Generation IssuesCertificate generation was delayed because of an upstream Let's Encrypt incident. This affected provisioning of new TLS-enabled addons, BYOC clusters and domain certificates. Addon provisioning was reported working again by 23:02 UTC, and the incident was resolved at 23:36 UTC after no further certificate errors.AI-graded · some customers · Certificates
Why this grade
  1. RULES · SEV 3
    Sev 3 because Certificates had degraded performance
  2. CLAUDE · SEV 3 · SUBSET · MEDIUM CONFIDENCE
    Certificate generation delays blocked new TLS addon provisioning, BYOC cluster provisioning and domain certificates while Let's Encrypt had issues.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 18:55 UTC · investigating
    We are currently observing delays in certificate generation due to an ongoing incident with Let's Encrypt: <https://letsencrypt.status.io/> This will affect provisioning of new addons with TLS enabled, the provisioning of BYOC clusters as well as domain certification provisioning.
  2. 23:02 UTC · monitoring
    No recent API errors experienced and addon provisioning is working as expected. We're still closely monitoring Let's Encrypt API responses.
  3. 23:36 UTC · resolved
    No more certificate creation errors have been observed from Let's Encrypt. This incident has been resolved.
Jun 10, 202617mSEV 3
Issues accessing app.northflank.comUsers could not log in to or access the Northflank web UI for about 16 minutes; running workloads were unaffected.AI-graded · Northflank App
Why this grade
  1. RULES · SEV 4
    Sev 4 because Northflank App had degraded performance
  2. CLAUDE · SEV 3 · UNCLEAR · MEDIUM CONFIDENCE
    Users could not log in to or access the Northflank web UI for about 16 minutes; running workloads were unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 15:58 UTC · investigating
    We are currently investigating this incident.
  2. 16:03 UTC · monitoring
    We implemented a fix and are currently monitoring the result. Users should now be able to log in and access the UI. Running workloads were unaffected.
  3. 16:14 UTC · resolved
    This incident has been resolved.
Jun 10, 20266h 23mSEV 2
Europe - West (London): Intermittent connectivity issues impacting Postgres cluster managementPostgres addons in the London region showed intermittent connectivity errors ('Error communicating with DCS') from 08:00 to 14:23 UTC on 10 June. A recovery at 09:45 recurred. The provider said the cause was an ongoing incident at its cloud provider in the region. No issues were seen after 11:05 UTC, and the incident was resolved at 14:23 UTC.AI-graded · regional · Addons
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE
    Postgres addons in London had intermittent connectivity and cluster-management errors for about six hours; other regions were not reported affected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
5 updates
  1. 08:00 UTC · investigating
    We are currently investigating this incident. You may see logs in Postgres with the following error message: ``` ERROR: Error communicating with DCS ```
  2. 09:59 UTC · monitoring
    We have observed a recovery starting about 20 minutes ago. Currently, Postgres addons are stable.
  3. 10:19 UTC · identified
    We saw a period of recovery between 09:45 and 10:04 UTC, but the issue has since recurred. This is tied to an ongoing incident affecting our cloud service provider in the region. We are continuing to monitor and will share further updates as we have them
  4. 11:53 UTC · monitoring
    We have not observed any further issues since 11:05 UTC. We are awaiting confirmation from our cloud provider that the issue is fully resolved on their side, and we will update this status accordingly.
  5. 14:23 UTC · resolved
    This incident has been resolved.
May 13, 20261h 26mSEV 3
Europe - West (London): New resources not able to be createdNew addons, jobs and services could not be created in the London region for about 90 minutes; existing workloads kept running.AI-graded · regional · Addons +2
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 3 · REGIONAL · HIGH CONFIDENCE
    New addons, jobs and services could not be created in the London region for about 90 minutes; existing workloads kept running.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 15:00 UTC · investigating
    We are currently investigating this incident. Existing running workloads are unaffected.
  2. 15:33 UTC · monitoring
    We are seeing an improvement; workloads are starting to be created. Connectivity has been restored, and we are continuing to monitor the situation.
  3. 16:27 UTC · resolved
    This incident has been resolved.
May 10, 20262h 02mSEV 2 · SECURITY
Active DDoS Attack in Europe - West - NetherlandsPublic ingress in the Europe-West Netherlands region was disrupted by a large DDoS attack for about two hours; internal traffic was unaffected.AI-graded · regional · Addons +1
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE · SECURITY
    Public ingress in the Europe-West Netherlands region was disrupted by a large DDoS attack for about two hours; internal traffic was unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 18:30 UTC · identified
    We are currently experiencing a large DDoS attack in the Europe - West - Netherlands region. This is affecting public ingress into the cluster. Internal traffic is not affected. The team is working on additional mitigations.
  2. 20:10 UTC · monitoring
    We have put in place mitigations and are monitoring the affected region.
  3. 20:31 UTC · resolved
    This incident has been resolved.
May 8, 20262h 36mSEV 3
Let's Encrypt Certificate Generation IssuesCertificate generation failed or delayed for new addons, domains and BYOC clusters during a Let's Encrypt disruption; existing services not described as affected.AI-graded · Certificates
Why this grade
  1. RULES · SEV 3
    Sev 3 because Certificates had degraded performance
  2. CLAUDE · SEV 3 · UNCLEAR · MEDIUM CONFIDENCE
    Certificate generation failed or delayed for new addons, domains and BYOC clusters during a Let's Encrypt disruption; existing services not described as affected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 19:00 UTC · identified
    [Let's Encrypt is currently experiencing a partial service disruption](https://letsencrypt.status.io/) leading to certificate generation delays and failures on the Northflank platform. This will impact provisioning of new addons, domains and BYOC clusters.
  2. 21:36 UTC · resolved
    Certificate generation is functioning normally again. We will continue to monitor the situation.
Apr 8, 202652mSEV 2
Addon and volume availability issues in US - CentralA node failure hit some stateful workloads, addons and volumes in US-Central for about 52 minutes; other regions were unaffected.AI-graded · one location · Addons +1
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · LOCAL · MEDIUM CONFIDENCE
    A node failure hit some stateful workloads, addons and volumes in US-Central for about 52 minutes; other regions were unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 06:15 UTC · identified
    We are aware of an issue with a node affecting some stateful workloads in US - Central. We are working on a fix for this incident.
  2. 07:07 UTC · resolved
    We have recovered the node, and all workloads are now running again.
Mar 25, 20263h 21mSEV 2 · SECURITY
Active DDoS Attack in Europe - West - NetherlandsA large DDoS attack disrupted public ingress to services and jobs in the Netherlands region for about three hours; internal traffic was unaffected.AI-graded · regional · Addons +2
Why this grade
  1. RULES · SEV 2
    Sev 2 because Addons had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE · SECURITY
    A large DDoS attack disrupted public ingress to services and jobs in the Netherlands region for about three hours; internal traffic was unaffected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 14:28 UTC · identified
    We are currently experiencing a large DDoS attack in the Europe - West - Netherlands region. This is affecting public ingress into the cluster. Internal traffic is not affected. The team is working on additional mitigations.
  2. 16:01 UTC · monitoring
    The region is currently stable. We are still actively mitigating the ongoing attack.
  3. 17:48 UTC · resolved
    The DDoS attack has stopped we will continue to monitor the affected region.
Mar 16, 202634mSEV 3
App UI is experiencing loading issuesThe Northflank app UI failed or loaded slowly for about 33 minutes due to database load; the text does not say how many users were hit.AI-graded · Northflank App
Why this grade
  1. RULES · SEV 4
    Sev 4 because Northflank App had degraded performance
  2. CLAUDE · SEV 3 · UNCLEAR · MEDIUM CONFIDENCE
    The Northflank app UI failed or loaded slowly for about 33 minutes due to database load; the text does not say how many users were hit.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 17:42 UTC · identified
    We are investing an incident where the UI is failing to load correctly.
  2. 18:15 UTC · monitoring
    We identified the root cause as a system component causing disproportionate database load. This is affecting read performance leading to degraded performance for the app UI. We have implemented a fix and are monitoring the situation.
  3. 18:16 UTC · resolved
    This incident has been resolved.
Mar 9, 202629mSEV 3
Degraded performance in US Central regionProvisioning new workloads and redeploys were delayed in US Central for about 30 minutes; running workloads had minimal impact.AI-graded · regional · Networking +2
Why this grade
  1. RULES · SEV 2
    Sev 2 because Networking had degraded performance
  2. CLAUDE · SEV 3 · REGIONAL · HIGH CONFIDENCE
    Provisioning new workloads and redeploys were delayed in US Central for about 30 minutes; running workloads had minimal impact.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 12:06 UTC · investigating
    We are currently investigating this incident.
  2. 12:35 UTC · resolved
    The root cause has been identified as an issue with the GCP infrastructure control plane and has been resolved. There was minimal impact to running user workloads. The main impact was a delay in provisioning net new or redeploying existing workloads.
Feb 19, 20261h 34mSEV 3
Issues with builds using the Heroku 24 BuildpackBuilds failed to start when using the Heroku 24 buildpack image; other build paths were not described as affected.AI-graded · one model or feature · Builds
Why this grade
  1. RULES · SEV 3
    Sev 3 because Builds had degraded performance
  2. CLAUDE · SEV 3 · SINGLE · MEDIUM CONFIDENCE
    Builds failed to start when using the Heroku 24 buildpack image; other build paths were not described as affected.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
2 updates
  1. 15:00 UTC · identified
    We are aware of an issue where builds fail to start when using the Heroku 24 buildpack image. We are currently working on identifying a solution. We will provide an update when the fix has been released.
  2. 16:34 UTC · resolved
    We have released a fix for the issue.
Dec 11, 202539mSEV 2
Node infrastructure stability - London RegionNode instability in the London region caused partial outages for some workloads, including services, networking, add-ons and jobs. A node image security release made the host filesystem read-only for certain workloads, making nodes unresponsive. A mitigation was applied and the region was operating as expected by 14:13 UTC. A permanent fix is still pending.AI-graded · regional · Networking +3
Why this grade
  1. RULES · SEV 2
    Sev 2 because Networking had degraded performance
  2. CLAUDE · SEV 2 · REGIONAL · MEDIUM CONFIDENCE
    London region nodes became unresponsive, causing partial outages for some workloads services, networking and add-ons for about 40 minutes.
  3. Claude Haiku 5.5 · prompt v11 · Oct 31, 2026 · Raw record
3 updates
  1. 13:54 UTC · identified
    There are currently issues with node stability in the London region leading to partial outages for some workloads. The team has identified the issue and is implementing a mitigation.
  2. 14:13 UTC · monitoring
    The mitigation is in place and the region is operating as expected.
  3. 14:33 UTC · resolved
    The incident has been resolved and the cause identified. A node image security release lead to the host filesystem going into a read only mode when interacting with a specific set of workloads causing the node to become unresponsive. We have implemented a mitigation and are working on a permanent solution.

Peers, 90 days

Compare

README badge

Northflank's live grade, for your README.

Northflank grade badge
[![northflank on downagain.dev](https://downagain.dev/badge/northflank.svg)](https://downagain.dev/p/northflank)