Versions Compared

Version Old Version 4 New Version Current
Changes made by

Jeremy Busk

Medha Parlikar (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

When: Jan 2nd, 2018: 10:07 am
Why: RChain was plugged on CoinMarketCap by Ian Balina - and we received over 18K new visits to the site.
Technical Cause: Still looking into but .. either high burst of traffic triggered event in intrusion mitigation system and a back-off period began or high iowait existed on host due to lack of resources.

Event log

10:06 am: Kitblake reports in Discord #admin channel that a Tg user is reporting that Rchain.coop is down and confirms outage. Notifies Patrick727

10:07 am: Nash Foster sends telegram notification to Medha Parlikar (Unlicensed) : RChain website down

10:20 am: Patrick727 confirms that the site is back up.


Info


site: https://www.rchain.coop/
squarespace.com uses a load-balancer or ADC(Application Delivery Controller) (probably NGINX). I would think it caches site in ADC and plays it from memory but maybe not

I can't see any global outages at that time
https://status.squarespace.com/

but maybe it just got reported later as in excerpt from page

...

Images Not Loading and Sites Loading Blank
Resolved - This incident has been resolved. 
Jan 2, 13:37 EST
Monitoring - A fix has been implemented and we are monitoring the results. 
Jan 2, 13:26 EST
Investigating - We are currently investigating an issue with some images not displaying correctly. 
Jan 2, 13:00 EST

...

Mitigation & Avoidance

Never put anything into rchain ecosystem without monitoring from a centralized source of truth.
Coordinate in future press releases to pre-allocate extra resources (automated scaling can take a little time) and have watch performance monitoring closely for fast reaction if their is an issue. 

...