Python In The Shell: The STEEMIT Ecosystem – Post #100
BLOCKCHAIN AND DATABASE MANIPULATIONS USING PYTHON
In continuation as per the previous post here:
https://steemit.com/blockchain/@lightingmacsteem/7ta7cg-i-t-spices-the-linux-way
We now discuss the code line by line:
1 ###CHECK THE DESTINATION FILE
2 dd = ('sed -n "$=" ' + destfile)
3 checkfile = (os.popen(dd).read()).strip()
4 if (str(checkfile) != str(0)) and os.path.isfile(destfile):
Lines 1 to 4 is a check if the temp file as represented by the variable destfile is present to make sure that the python script will not exit prematurely. Lines 2 and 3 being the last of the routines that uses the linux shell command sed.
5
6 ###BE SURE TO OPEN FILE IN UTF-8
7 list = open(destfile, 'r', encoding='utf-8')
Lines 5 to 7 is the part where the file being present is opened for reading thru the list variable, effectively placing the whole 300 thousand JSON lines into memory. We tell python to process the characters inside the said file as utf-8 to ensure ourselves of an exact character read/writes and therefore eliminate processing errors.
8 print('There is/are new blocks, I will update the database with about ' + checkfile + ' new records; please wait.......')
9 print('There is/are new blocks, I will update the database with about ' + checkfile + ' new records; please wait.......', file=open(logfile, 'a'))
10 time.sleep(9)
Lines 8 to 10 are message lines, line 8 being displayed to the monitor screen and line 9 to make sure the exact message is written to a log file for later processing. Line 10 is just a wait line, in seconds. Here the python code will stall and take a breather for about 9 seconds.
11
12 ###LOOP LINE PER LINE HERE USING WHILE
13 for line in list:
14
15 ###WHOLE BLOCKDATA HERE UNALTERED
16 blockdata = str(line).rstrip()
Lines 11 to 16 is now processing the very lines as can be seen thru the list variable. We have to remember that one line means one block data, and whatever it may be carries data that may prove very important in future processing.
The for line is a loop command in python, which simply means that all lines in the list variable will be processed line by line using the exact and same routines as indented. This will enable us to segregate the blockchain data, each data here will mean a column entry in the database if you can picture what I mean.
The next post will explain the lines of segregated data as extracted by python from the JSON lines of the blockchain temp file, said data will be the very data to be inserted into the database.
Until next time.
“Segregate Your Pills, Same Color Does Not Mean Same Purpose…….”