Running a Witness: Is it as Easy as it Sounds? πŸ€”

in #witness-category β€’ 7 years ago (edited)

It's been a month and a half since I set up my witness node. I can tell you one thing, it's not as easy as it may sound. I've seen numerous posts encouraging people to set up a witness server, with the promise of making big money out of it. The truth of the matter is, running a witness is not a plug-n-play process. It's a responsibility, not a game. It requires personal financial investment, dedication, and most importantly technical skills. Let's go into some details to clear out some of the misconceptions.

This is not what being a witness is all about
(we wish it was, but it's not)



This is



The Hosting

To run a witness, you need to have your servers hosted somewhere (VPS, dedicated, colocation, whatever suits your budget). Yes, servers, we'll get to that in a minute. The steem daemon is a very finicky process, it requires lots of RAM, low network latencies, and preferably no other running process on the box. The first server to set up is the witness node. It is the main powerhouse, with two roles: host the blockchain and create the blocks (the steem transactions). The witness IP is kept secret, for security reasons. The second server is the seed node, it's a publicly available server with the role of distributing the blocks. The more seeds spread across the world, the better, because they help reduce the latencies. The third type of server is the full API node; witnesses are not required to run one, but if they chose to, this monster requires much more RAM (or a fast RAID SSDs), i.e. more expensive.

The second consideration for a successful setup, is to have a fail-over mechanism, so in case something goes wrong with the node, you can have a backup server to keep generating blocks for you, or you will be missing blocks, which is bad. Of course, there's the option of disabling the node's signing key in your steem account when things go sour, but the downtime can cost you precious blocks.

As you may know, witnesses are rewarded with Steem Power for every block they generate. It's around 1 SP (last time I checked it was 0.948 SP). The Top 20 witnesses are rewarded 1/5th of that amount, so ~0.190 SP, because they generate more blocks per day. I won't get into the details of the witness economics, there's plenty of other posts that covered that. Needless to say, to be successful, you need to generate enough rewards to cover your monthly expenses, or you will operate at a loss.

And finally, securing your network goes without saying πŸ”’. In fact, it's important to secure it. You don't want to be hacked (the configs contain private keys) and you don't want a brute force attack on your port 22 that brings down your server, do you? In fact, changing it helps reduce missed blocks.

The Votes

Running the server(s) is one thing, the other crucial consideration is the rank. When people vote for your witness, their VESTS (Steem Power) are taken into account. The amount of votes you receive is irrelevant. For example, 1 person's vote with 1000 VESTS is equivalent to 1000 votes with 1 VEST each. So if you have a thousand minnow friends ready to support you, the guy next to you with a few whale votes will eclipse your rank. The rank is important, the higher it is, the more probability you have to generate a block on the next scheduled round. For example, I'm at rank 56, which gives me about 15 blocks/day, while someone in the Top 20 generates ~1371 blocks/day (with 1/5th of the reward). As you can imagine, people at the top have way more pressure to run a reliable setup than those at the bottom. At worst, I could miss 15 blocks/day, but for someone at the top 20, that would be 1371 blocks if there's no solid backup(s) and fail-over mechanisms.

The Dedication

After you set up the server(s), and got the votes to be in a comfortable rank position, what next? Well, depending on your projects and involvements, keep it up. However, blogging and running projects can eat a lot of one's time. You also must assume your witness role by hanging out with the other witnesses! If there's a generalized issue occuring (like the bandwidth problem) and an update coming up (like 0.19.1), you need to be in touch and be up to date with the rest, not spending your time on a cruise ship. Reaction time is a factor, if you can't react when needed, you'll lag behind, possibly affect the rest of the network, and ultimately lose your rank.

Of course, reaction time is also important when managing your setup. For example, last week my witness server crashed for some mysterious reason, it was on the provider's end, because a few other witnesses crashed too. I was quick to react and manage things, which prompted me to improve my set up a few days later. Other witnesses couldn't do that, for a lack in the things I discussed. Surprisingly, a top 20 witness also experienced simultaneous failures and a lot of missed blocks, which demonstrates that nobody is immune to server problems. If it can happen to the best, imagine what can happen to a non-knowledgeable witness.

Part of the dedication is the constant learning. Therefore reading, interacting with others and understanding the system's operation is part of the witness job. Paying attention to the other witnesses concerns and listening to the more experienced witnesses' wisdom is a source of knowledge that prevents future problems.

Conclusion

A lot of the new witnesses jump onto the witness bandwagon without being aware of the important requirements I mentioned, because nobody tells them about it. It's possible to run only the witness server, but if you do, you risk more when bad things happen. I threw a lot of technical words on purpose because this wasn't meant to be a newbie guide. If you don't understand what I was talking about or don't know how to change dir on linux, then you should probably reconsider running a witness. Some people choose to hire a tech to manage their witness, which is fine, but it adds to your expenses. Reaching the top 100 can be done with enough votes for a newbie witness, banking 1-2 blocks a day. Aiming for the top 50 or the top 20 (even harder), doesn't and won't happen overnight. It takes time, patience and hard work. Are you up for the task?


Witnesses are the backbone of Steem, they host the blockchain in the background, and I'm happy to be part of that.


Proud member and delegator of the @minnowsupport project.
Join us on https://discord.gg/GpHEEhV

MSPPAL


If you like this post, upvote and comment.
If you really like it, resteem.
If you love it, follow.

Follow
πŸ—³ Check out my Steem witness application and vote for me. πŸ—³
Thank you.

Vote

Alternatively, you can issue this command in cli_wallet (after unlocking it)

vote_for_witness "YOURACCOUNT" "drakos" true true

Sort: Β 

I knew it wasn't easy but I had not idea it was this hard, thank you for doing what you do. I went ahead an voted for you.

Very nice written and well explained.

I would like to add to the backup witness server a note: If you set up a backup witness, choose a different provider. Even better, at a different country. If you host both witnesses (main & backup) on the same provider, they may end up in the same host node (if you are using VPSes), or in the same datacenter. In case of a blackout (power/connectivity/whatever), you are simply paying more to have nothing.

The same goes for the seed node. Do not host it on the same provider. If possible, host it somewhere as far as possible.

And there is always the "risk" of 2 providers using the same datacenter for operations, and they only rent 3-4 racks. You can end up using the same datacenter or even room for your witness/seed operations.

Post resteemed to raise awareness :)

Yes indeed. I didn't go into too much details, I wanted to give a global view of the witness challenges.

Β 7 years agoΒ (edited)Β 

KEEP AT IT CAPTAIN! YOU'RE A WITNESS IN MY BOOK.

Β 7 years agoΒ (edited)Β 

Exactly. There is more involved that what I had anticipated when I started my witness. Nothing outside of my ability, but it has been outside of my comfort zone. I can say that with your help, and the campaign, I've strengthened my skill set and I appreciate that. Had I known all of what you mentioned above BEFORE I decided to throw my hat in, I would have probably not started it. It IS an investment, and it is NOT just plug and play as you state *edit (bad wording.. You did not state that it was easy, that was meant to be agreement on your statement). Though I can say this it has entrenched me even deeper into the platform to make it the best I can from my position. Cheers on the post @drakos!

I've helped (freely) many newbie witnesses with their setup. But deep down it pains me to seem them embark on this enterprise when I see their insufficient technical skills. I've been wanting to write this post for a while, so here it is. At the risk of sounding a bit tough in my writing, I think we need a dose of realism.

Now this is how you serve a well needed dose of realism. Too many people swimming beyond their depths. I am relatively new to steemit and if you don't mind I have set you as my proxy for voting witnesses as it appears that you know better what to do with the votes than myself. Thanks for this reminder on the importance of due diligence.

Great article. I was curious about all this. Not surprised it requires a lot of dedicated time and technical skills.
About the huge amount of RAM, that is often a sign of something that needs to be optimised in the software code... Maybe it is a known issue ?
And .. hehe I am glad I voted for you then, even if my vote is nothing compared to most here.
! :)

Every vote is important ☺️

can i vote for you as a witness?

Of course you can. The instructions are in the footer of my posts, and I wrote a guide about voting for witnesses.

Β 7 years agoΒ (edited)Β 

Thanks for the info, I plan on running a witness one day. I am going vote for you now.(looks like I had already voted for you) lol

Your vote is appreciated πŸ˜€

If anyone doesnt want to read all here is a summ up:

Beeing a witnes is a pain in the azz. If your azz is soft - dont do it.

Now thinking bout it....do witnesses have firm bottoms? :3

Β 7 years agoΒ (edited)Β 

Well put. Care to examine mine?

Bend over and think of daisies. :>

Β 7 years agoΒ (edited)Β 

You got my vote sir. Resteeming <3

I humbly accept it πŸ™‡

Β 7 years agoΒ (edited)Β 

It is good information, I find that many people are like witness what's that? So maybe posts like this will help clear that up a bit.

Congratulations! This post has been upvoted from the communal account, @minnowsupport, by drakos from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, someguy123, neoxian, followbtcnews/crimsonclad, and netuoso. The goal is to help Steemit grow by supporting Minnows and creating a social network. Please find us in the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.

This post has received a 0.45 % upvote from @drotto thanks to: @banjo.

Great post! This should be pinned in the Steemit chat #witness channel :)

Tell that to gandalf 😁

Woow! I know very little so far about Steemit, but you have an incredibly difficut work to do. Good on You!

And for that reason, you have my vote . Doing great things for this community and we love you for that !! Keep up the good work

Thank you for the kind words.

I look like her minus the purple hair πŸ˜‚πŸ˜‚πŸ˜‚πŸ˜‚πŸ™ŠπŸ’œ my neighbor recently handed me a book on Linux and now that you mentionned it, will definetly look more into it! Keep it up babe

Β 7 years agoΒ (edited)Β 

Une montrΓ©alaise en plus! Cool. You have an interesting blog, I will be checking it for sure.

OH MY GOD SAY WHAT HOW DO YOU KNOW IM A FRENCHIEπŸ˜πŸ˜‚πŸ˜‚πŸ˜‚ you speak FRENCH too?? Sorry I'm freaking out there's no frenchy pants on here it's awesome to see you're from mine πŸ˜πŸ™ŠπŸ˜ thanks cutayyyπŸ’œπŸ’«

Β 7 years agoΒ (edited)Β 

Ben, j'ai lu ton blog et ton introduction. Oui, je prend le temps de répondre aux gentils commentaires ET de découvrir le blog de leur auteur. Vive la poutine de la belle province 😁

Wow t'es le bestest πŸ’—πŸ‘ŒπŸ» Merci full!!!:) comment Γ§a tu parle franΓ§ais mister?πŸ˜πŸ™Š

ca-fr comme toi πŸ˜‰

Dayum broπŸ‘ŒπŸ»β€οΈπŸ˜‚

Upvoted for tge good and educative content

Interresting article, thank you! I haven't myself considered becoming a witness but haven't either really known about how it is working. This article gave me a lot of new information of what it is about and also confirmed that I would need to learn a lot more before I could consider becoming a witness. Thanks also for the contribution you do for the steemit society, I hope it is an apreciated one!!

Thank you !!
Namaste

Is that @jerrybanfield in drag on the first image?

hahahah, no.

:D haha

Congratulations, you just made me cancel my witness proposal!!

Great job man, you give a very good explanation for what witness is in another point of view!

so question about ranking, the higher rank the more blocks you can process, and only way to increase in ranks are voting?

This is an awesome post!

In fact it contains things that only one of your closest friends or family will tell you. While some posts do mention hidden somewhere that it could be tough, you have put down all the details of the areas that would require consistent effort upfront.

There has to be realism attached to becoming a Witness - I see too many newbies are rushing in after signing up, in most cases not even knowing what Steemit is all about. Agreed, the two require two different skillsets but a new signup rushing in for the money is a sure setup for failure - especially when there are so many make money options out there.

I have a question - does one require to have a very thorough knowledge of Linux to operate as a Witness, or just being familiar with the basics is enough?

Thanks.

Β 7 years agoΒ (edited)Β 

The guides written about the witness installation do not cover how to secure your server, customize the scripts (bashrc, fail-over, for example). The linux basics may get you around, but you need to have some additional knowledge to do those extra things. To give you an idea, most of the top 20 people are professional developers and IT people.

Congratulations @drakos! You have completed some achievement on Steemit and have been rewarded with new badge(s) :

Award for the number of upvotes received

Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here

If you no longer want to receive notifications, reply to this comment with the word STOP

By upvoting this notification, you can help all Steemit users. Learn how here!

you are 100% right, we chatted in witness chat room!!!I have y witness still running, but haven't taken it to online ... its a lot of additional time spend to maintain it.have you had chance to produce blocks yet?

Yes I'm producing blocks.

Hi ! This post helped me understand a couple of things about witness.
Voted for witness upvoted resteemed.
Yet i didnt understand how i can vote for more than 1.
tip!

You just vote for more witnesses, to maximum 30.

Hi @drakos! You have just received a 0.1 SBD tip from @utzo!


@tipU quick guide | How to check your pending payouts.

After reading this I wanted to vote, but then I noticed I’d voted already, lol. Take care.

I've personally wondered how much time you dedicate to this project as you're always active on Chat in pretty much every room I've seen.

Are you living off this or somehow balancing it with a job?

I'm pretty much fulltime on this.

Enlightening. Upvoted,resteemed and followed.

Β 7 years agoΒ (edited)Β 

I salute you, man. I know you have a lot on your plate right now. It takes a lot of gut to do what you're doing. I have a question, can I vote multiple witnesses? I already voted you... :))

Β 7 years agoΒ (edited)Β 

@pinaynomad, thanks for your vote. If you look at the top of the witness voting page, you can see that you can vote for 30 witnesses at the same time. The votes aren't permanent, you can change them anytime you like.

thanks mate! Looking forward to your future projects. :-)

I eventually want to become a witness, and this gave me a very good idea of what to expect. Thank you very much for this post.

I think I will be able to do it, I do have the technical skills needed to manage a Linux server, I think i'll have a bigger problem getting the votes necessary to make it financially sustainable.

what i miss is the amount of bandwidth that is used by the servers can you host a backup server at home if you have optic fiber ?

You can host it anywhere you want, as long as you have a reliable 24/7 low latency network. There's a witness running it from his house, but he often misses blocks. It seemed to be related to his ISP. Another witness ran some of his nodes on OVH in past, and he said it was crappy for running nodes on it as him too missed blocks. If you want, you can try a seed node setup (no signing key required) and see what kind of latencies you get when it's synchronized. They vary between 35-550 ms. Also, the network speed (gbit and more) is more important for a full API node.

i might try to setup something like that in my new house somewhere in october , already have virtual linux running so it shouldn't be to much work to test things

So basically you earn about 15sp a day for being a witness? Because you receive about 15 blocks a day.
How much money have you invested in the equipment?
Thank you very much for sharing your experience

Altogether it costs me around $100 per month for the servers, plus all the time being a witness.

My mind just exploded... I made a small attempt to mine steem, but failed. As I understand only witnesses can do it? Not like with other currencies?

Mining was removed from steem a long time ago. Yes, witnesses are the ones responsible for generating the blocks.

Can I get some upvote for some of my post @brainy723

No. Please don't ask that in comments, it will get you flagged.