Custom JSON Attack
This week, we experienced a new type of spam attack that hindered the normal functioning of the Steem blockchain servers: seeds, witnesses and RPCs. I had written a script to monitor all my servers and got an alert as soon as the incident started. I immediately scrambled to investigate, it was late in the afternoon.
It began with unusually high latencies on some servers, which corroborated with what other witnesses were experiencing. Soon enough, a spammy custom json started to show up in the logs à la Matrix, while causing many other transactions to expire. I won't discuss the details of this in order to not give ideas to malicious users. For some reason, some servers couldn't cope well, then stalled after being hit with the repetitive spam. Other servers weren't affected at all.
Being available, for hours after the incident started, I was able to closely monitor and manage my servers to restart them as needed when/if they stalled.
Steemit developers were notified and within 24h they provided a fix. However, before the release of a final stable fix, I helped in testing it on some of my servers. When the stable version was done, I was promptly and fully updated, without interruption to my block production. Many witnesses also updated to mitigate any future attempts of this attack type. While the working fix was still under the 0.19.2 version, an additional tiny PR was added later to indicate version 0.19.3. I'm proud to say (without bragging) that I was the first one to upgrade to the final 0.19.3 version :)
Thanks to the Steemit team for quickly dealing with this issue, as they always did with critical incidents. And thanks for the witnesses who helped in testing the patch.
If you're a witness using the dockerized setup and haven't patched your servers yet, you can check this post https://steemit.com/witness-update/@someguy123/important-update-steem-in-a-box-for-v0-19-3-steemfeed-js-update
Servers Tinkering
For the last month, I've been busy tinkering with different RAM, zRAM, swap configurations on different VPS providers and at home. In fact, I recycled my old computer and rebuilt it for local testing. I will make a more detailed post about it later. Rest assured, my main witness node is a dedicated server with 64GB RAM that has been running quite well so far. Currently, I have 5 servers in total, for a higher backup redundancy, with different configurations and locations, including the seed.
Note that the blockchain file is now over 100GB (and keeps growing).
Witness Parameters
- My APR and bias are at 0%.
Miscellaneous News
- I'm currently at Rank 25. Thanks to all my supporters. Only 7 missed blocks since I started witnessing 9 months ago.
- I've been actively engaged with @the-resistance since it started, to counter and expose @grumpycat aka @madpuppy abuses. Join us if you'd like to participate in our cause.
- I'm still a member and delegator of the @minnowsupportproject. I recently bumped my delegation to @youarehope.
- I'd like to thank @wackou for his continuous support for me and his generous increased delegation.
- I continue to help users and witnesses on Steem.Chat.
- There's been an improvement in the stuck accounts, since the new Steemit signup faucet was implemented in March 2018. These are the stats of the users I processed on Steem.Chat:
| Month | Cases |
|---|---|
| Nov 2017 | 41 |
| Dec 2017 | 126 |
| Jan 2018 | 373 |
| Feb 2018 | 784 |
| March 2018 | 474 |
| April 2018 | 136 |
| TOTAL | 1934 |
Security Reminder
Phishing websites are still around and users are still falling for it. Sometimes more than once!
Use the Posting key to login for your daily blogging activities.
Here are my commandments:
DO NOT USE YOUR PASSWORD AT ALL, FORGET YOU EVEN HAVE IT
DO NOT PASTE YOUR PASSWORD ANYWHERE OUTSIDE STEEMIT.COM
DO NOT GIVE YOUR PASSWORD TO ANYONE
STORE IT OFFLINE AND DO NOT LOSE IT
Use the password ONLY for doing critical tasks, like account recovery or keys change.
Check out those quick guides about the different keys and the security of your account:
Together we make Steem better.
Check out Steemian.Info, the concise information resource about the Steem blockchain
Proud member and supporter of @the-resistance
Join us on https://discord.gg/qMWCbWR
Proud member and delegator of the @minnowsupport project
Join us on https://discord.gg/GpHEEhV
Available & Reliable. I am your Witness. I want to represent You.
🗳 If you like what I do, consider voting for me 🗳
If you never voted before, I wrote a detailed guide about Voting for Witnesses.
Go to https://steemit.com/~witnesses. My name is listed in the Top 50. Click 
once.
Alternatively you can vote via SteemConnect
https://v2.steemconnect.com/sign/account-witness-vote?witness=drakos&approve=1
An hour or two after you posted this, there was another attack or however you will choose to call it, and I could not even transfer funds.
You got a 14.60% upvote from @luckyvotes courtesy of @stimialiti!
You got a 13.61% upvote from @sleeplesswhale courtesy of @stimialiti!
You got a 22.73% upvote from @proffit courtesy of @stimialiti!
Send at least 0.01 SBD/STEEM to get upvote , Send 1 SBD/STEEM to get upvote + resteem
Thank you for being a stand up witness with integrity. Supporting minnnows and defending against injustices.
Gracias!. Bravo!. Well done. Thank you for the efforts you put in to bring about normalcy back to the block chain. Really appreciate all you do. Your efforts is obvious for all to see and thanks to the creators of Steemit who provided the fix. Together we can take Steem to the moon. Massive respect for all you do.
Dear Friend @drakos , You are doing great tasks to monitor servers and help members, I am really proud of you and your friendship, and you should also be proud that you are the first to monitor and resolve the problem, in addition to you have update the final version 0.19.3.
You do great things, we are proud to see you, and you deserve to be the highest list of witnesses because you deserve the best .👍
Good luck my dear friend @drakos 👍👍😃
All the best for you 👍👍👍👍👍😃😃😉
Stirling job sir Drakos.
Just read your voting for witness guide and that definitely answered a few questions I had.
For non techy Steemit members it's all too easy to be unaware of the work that is done behind the scenes server end, that keeps this platforms integrity together.
So thank you for your hard work, you have our witness vote!
Though I had no doubts, now I know for certain – you more than deserve my witness vote.
It's easy to forget just how much work you guys are doing behind the scenes. I sincerely appreciate reading such a detailed account – even if half of it is not so unlike reading Martian.
You're doin' a damn fine job, sir @drakos. Tipping my hat in your general direction. ;)
Job done
🙇
Thanks for the update @drakos. I really appreciate the witnesses that do this. I don't understand much of what you're saying about servers but it's enough to tell me that you are on the ball! 😁
It's tech mumbo jumbo, but believe me, it works 🤓
Good enough for me! 😁
Great job! Proud to have you as a witness. you deserve to be in Top 20 and hopefully you will.
Thanks for the update! We appreciate the transparency
Thank you for the new updates that have been downloaded
I voted for you as a witness because you are someone who helps everyone
Invite everyone to vote for you
You really are a successful and wonderful person here
You have all my love😍😍😘
thank you for tell us about witness.supprot minnows.best of luck.
A debt of gratitude is in order for the refresh @drakos.
I truly welcome the witnesses that do this. I don't see quite a bit of what you're saying in regards to servers however it's sufficient to disclose to me that you are on the ball!
Thanks for informing us about witness update
Can't JSON attack a real pirate
No way, I'll slice them with my cutlass.
Nice update thanks @drakos good to know the servers are in capable hands.
Wow good that you witnesses exist, it is good to know that to any malicious attack you will be in the vanguard to eradicate it.
You are super valuable to everyone in the community, thank you! ;)
Thanks for all you do and for making these updates. I am not techy at all, but I am slowly catching on to some of it thanks to posts like this. This old dog CAN learn new tricks, it just takes a while.
Do you guys in the resistance only focus on Grumpycat or do you also deal with general abuse. Im asking cause i think you could have a much general effect on the platform then turning it into a feud againts one account. (if youre not doing that already)
He started this whole mess. Our beef is only with him. When he goes away and stops his pestering, we will stop.
Sure.. But you must see a further potential for it? Its not called resist Grumpy. Would be a waste if you only focused on one guy. At least thats how i see it.
you won a vote as witnees from me for always helping a friend here in steemit, thank you very much @drakos
Hi @drakos ! Can I ask you, what does your nickname mean? Is this name related to the dragon?