NETWORK DOWNTIME: POST MORTEM. The previous 24h where challenging to say the least.

22 Aug 2023, 09:06

NETWORK DOWNTIME: POST MORTEM The previous 24h where challenging to say the least. We faced severe network issues, caused by a scheduled (and announced) network maintenance in our main DC which has gone entirely sideways. There's no way of sugarcoating it, something like this hurts our miners, our community, our project and everyone involved and needs at all costs to be prevented. An out of our control mistake during this maintenance led to significantly exhausting the max. allowed count of prepared DB statements, which is something only our DB cluster operator is able to restore. Static statements have been working and we applied changes in the code to prevent this going forward, updates have been deployed to all pool operators. Having that said, this further increases the urgent need to even more redundancy as a first step (which I instructed already to be deployed). But more importantly we need to accelerate the process to remove mallob entirely from the miner software. Our team has been working already since while on embedding the PoUW dataflow seamless into the Stratum protocol which is commonly used with pools. Current ETA for testing is in around 4 weeks, we will put all our weight in finalising this asap. This means - that in the near future - miners won't need to connect to any mallob endpoints any more - there will be no more mallobservices. This will also entirely remove some ongoing mallob stability issues some miners have been facing. Looking at our plans and roadmap, we need a reliable network for the commercialisation of our platform. We believe that the measurements and development steps taken will be leading us there. We also want to emphasise that we have been very clear from the beginning that Dynex is still work on progress - we need to build out the ecosystem, and that takes time and effort - given what we are doing has never been done before. The results of our work are very promising, the feedback from our beta users is beyond our expectations. There's a lot to come. Finally, we are also conducting a formal review of the pool mineradnow, which apparently was the only pool being able to communicate through static prepared statements during this phase. As transparency is very important for us, we will issue the results of this investigation and potential measurements in due time. Having that said, we apologise for any inconvenience and really appreciate your continuous support!