02:23 PST We are investigating a network switch failure affecting customers located on nodes v77XX
02:42 PST From the messages in the switch console it looks like the switch ran out of RAM. Upon further checking, this is likely related to a bug in JunOS.
We are restarting the switch to bring it back online.
03:07 PST Switch is back online and connectivity has been restored. We are continuing to investigate the root cause of this outage.
03:24 PST Switch is showing more memory related issues. After further investigation it looks like this is a JunOS-related memory leak. We are applying a JunOS update.
Additionally, we are preparing a spare switch in case we need to replace.
04:13 PST Connectivity has been restored. We are continuing maintenance work to make sure everything is working as expected.
04:22 PST No more issues have been found or observed. JunOS update completely resolved this issue. We will be closing this incident.