Updating the Database

avatar

keepdiggingminergemdiamond.jpg

Forgive me Father: For I have sinned!

So I was using MySQL to search for a post that I had written so I could link back to it... and it didn't pop up. Something about regulations. I know I had written it, but it wasn't there. Then I quickly remembered that it's been a while since I've updated my database, so I opened Workbench to see how long.

JUNE!?!

GAH! June? really? Haven't updated since June. So bad. Such a slacker. Terrible terrible terrible. Oh well! Back to the grind... now where was that script that imports my posts into the database... hmmmm. I checked a couple files where I thought it should be but it wasn't there. Then I checked my temporary testing file. The notation I use for temporary files is just 't', so this file was called t.js... I opened it, and the code was there... wow this is embarrassing. A critical function stored in a temporary file. Hm, k.

DatamineMining.jpg

The only times I have actually used an SQL database like MySQL (pronounced my-sequel but I say my-S-Q-L anyway like a noob) was first during college. That class was actually pretty useful (others not so much). It was one of those situations where the class split up into groups of 5 or whatever and we had a huge group project due at the end of the year worth a massive part of our grade. As we all know these things can turn out poorly with some people leeching and doing like none of the work, but this time it was fine.

My group got together and we all talked about what kind of product we should make. I can't remember other people's idea's but they weren't bad. However, I made a really hard pitch for a poker database, which was pretty uncharacteristic of me at the time. Usually I would just stay out of it and just go along with whatever, but as an aspiring poker player I really thought it would be a good idea if I could get everyone on board with the poker database idea, and it worked (because I was able to tell them exactly how it worked and they realized it would be the easiest path). If I recall correctly it didn't turn out exactly how I wanted it (because there were weird forced requirements like using a primary key auto-increment ID counter), but it did work and we did get a pretty good grade.

I worked on this a bit more years later, but after "Black Friday Poker" (Google it) the entire online poker industry got a bit gutted. Profitability went down and counterparty risk went up. Not a good combination. I stopped playing poker so much... and this was April 2011 so silly me I should have put everything into Bitcoin (oops!). I had certainly heard about it multiple times by then, and I thought it was interesting but just too complex to get involved with at the time.

image.png

What do you mean we can't claim we sold a billion golf balls instead of reporting our gambling establishment?!? This is highway robbery!

Hm yeah, so other than that poker database the only other time I've used SQL has been in the context of crypto and Hive, except this time I'm using JavaScript instead of Java... and I'm using the Hive API and HTML/CSS and Node.JS and whatever else. These cascading technologies can become overwhelming quite quickly for one person to wrap their head around, as you can imagine.

I actually created my own clone of Cards Against Humanity on Steem using Java in 2018. Back in those days, some random dev was puttering around and created wrapper code that allowed Java to make native JavaScript calls on Steem, which in turn allowed me to create the game because I didn't know JavaScript but I did know Java (kinda).

The game worked using comments on Steem as the way to play a card in your 'hand' (custom json wasn't even an option because of the wrapper). It was actually pretty funny... I created a 'bot' that played the game just by playing random cards, and sometimes the random answer was funnier than intentional ones. Go figure.

Long story short that wrapper broke for some reason (and the developer quit), which is really weird to me because our core codebase hasn't really changed at all, and I still have scripts from 2018 that still work once I switched over the JavaScript. Java breaking was pretty demoralizing so I quit for a while, as I recall.

Speaking of devs quitting, I'm still randomly amazed that this network is still alive, let alone doing well and grinding out code. If we look at the history of this community over the last six years, the journey has been absolutely abysmal on the "stable environment" scale. You can see for yourself in GitHub how many really talented devs have come and gone. We lost a big chunk when Larimer left because Ned was being a ridiculous dipshit. We lost a big chunk when Ned inevitably sold us down the river to a vulture capitalist. And we've lost good people here and there all throughout the gutter we've been crawling through. If we end up not getting thrown under the bus again over the next 2 or 3 years, we're going to end up making some massive progress like never before.

java javascript ham hamster.png

Just for anyone who isn't familiar with coding... Java may sound a lot like JavaScript, but they might as well be called two completely different things. In fact, Java is much much closer to C++ than it is to JavaScript. I actually like Java quite a bit, which is funny because most devs HATE it. I believe this hate stems from corporate America and has less to do with how Java actually works.

Java is a lot faster than scripting languages like JavaScript and Python, but it also requires you to be more specific with your containers and notation. Java has some features that make it a bit easier than C++, like the garbage collector (you don't have to allocate or deallocate memory to prevent leaks), but these factors also make it a little slower compared to a fully optimized C++ program. Hive Core is built in C++, so it is theoretically quite fast if properly optimized. I find this a bit odd because we never see any of this code even as developers on the outside. Everything is JavaScript interacting with servers and such.

image.png

shipinbottlebottleneck.png

That's the thing about the Internet

The Internet is slow and computers and servers are fast. Scripting languages like JavaScript are very good at what they do. In many scenarios they don't need to be fast or optimized because all they are doing is sending a ping to a server and waiting for a response. The vast majority of the wait-time has nothing to do with code optimization and everything to do with simply waiting for that server on the other side of the world to give you an answer. Once we factor in the fact that JavaScript is essentially the default code for interacting with HTML and webpages... it actually enrages me that learning it in college was not prioritized over Java, C++, and even assembly language. That's corporate America for you.

image.png

What was this about again?

Oh yeah... my critical post-import database code was in a temporary t.js file. Remember?!? Yeah anyway so I go to run this code and it scoops the first 100 most recent posts into my database using Node.JS... and then it just stops. Hm, that's weird it should have grabbed more. It's a recursive loop that should keep importing posts until it finds a copy of one already in there.

So I check the code for the bug and what I wrote was the most hackish nonsense ever. Future me was embarrassed for past me. Past me had just gotten the job done ASAP and never made a real solution for importing posts. And guess what? Rather than fix the code I just did some manual inputs to get all six months worth of posts in there. Attaboy!

Basically this code only works if the database is empty. I seriously considered deleting all the data and just reimporting everything but the thought of doing such a thing was so cringe I just couldn't do it. Ah whatever it's all back up to date again, for now. This time if I remember to run that script before it's 100 posts out of date I won't have to do the manual importing. Or I could just, you know, make it actually automated and work for reals. Nah! Life of a dev. Sort of.

Posted Using LeoFinance Beta



0
0
0.000
25 comments
avatar

Think learning Java and C++ in school was bad... I was taught COBOL and FORTRAN... on punch cards. Then years later had the ground-breaking opportunity to use IBM's DB/2 database. Revealing my old age here, and my how far we have come.

Very cool to meet someone else here wrangling up some code! And temp files, I do the dele, dele1, dele2, ad infinitum. Then after a year or so delete them all seconds before I realize I needed sample code out of one.

Great read, as I do something very similar. I import to MariaDB (a fork of MySQL) but use Lucee CFML as my language. It runs on the JVM (through Tomcat), but you don't have to write Java. CFML is very fast but Old Skool. Most operations, including importing the last 1,000 records from a HIVE API call takes typically 0.01 seconds to parse and INSERT into MariaDB. I also have a separate panel to query the database. All done as a web app in HTML/CSS and touches of JavaScript for client interface trickery.

0
0
0.000
avatar

I was taught Fortran and Pascal in school through on the chalk board, maybe it’s why i didn’t learn anything

Posted Using LeoFinance Beta

0
0
0.000
avatar

Chalk board much faster than my one finger, "chicken peck each letter", style of typing back then. It would take a good two hours to punch a 40 line (40 cards) program in, due to at least an equivalent 40 bad cards thrown away. One mistake, punch the whole line/card over. I laugh and think about it as a WTF situation now.

Plus, having to personally buy the blank cards. I'd like to say those were the good ol' days, but logistically, it was a nightmare all around.

0
0
0.000
avatar

That F.B.I. notice has five typos.

Yes, I’m a nerd.

0
0
0.000
avatar

lol nerd rage
that day sucked

0
0
0.000
avatar

I only found 3, maybe 4, so I acknowledge @preparedwombat is the superior nerd.

Posted Using LeoFinance Beta

0
0
0.000
avatar

F.B.I (missing the third period)
obtainedby (should be two words)
Issued (no need for capitalization, this is not German)
theUnited (should be two words)
orowning (should be two words)

Many years ago, I was employed as a proofreader. I can’t not see typos.

Except in my own posts/comments. 😅

0
0
0.000
avatar

And that's exactly what it said back in the day as well eh?
So unprofessional lol.
wtf

0
0
0.000
avatar

LOL!
I missed those.
I have the same problem with my typos. I think my brain knows what it is suppose to say and glosses right over my errors..
:)

0
0
0.000
avatar

Lol. Blame the typewriter or secretary. Better still anyone you find 😄

0
0
0.000
avatar

LOL.
It was surprising to see 3 typos, maybe 4, I missed the fifth. This is a huge, impressive public prosecutor office in New York.

Posted Using LeoFinance Beta

0
0
0.000
avatar

CS courses are so out of date, i expect some universities to still be using punch cards.

But really, Javascript was just the right tool, at the right time to work with CSS and HTML.
And like, now, those three things are so interconnected that it is hard to define what "HTML" actually does.

It is funny how so many things have risen to take away the top spot from C/C++ (and C++ should die a horrible death) and i have watched many contenders come and go, and C continue to work.

And the story of Javascript (the name) is sad, in a funny way. They were trying to use the excitement going into Java to launch the script.... and... almost died because of stupid MicroSlothish bullshit trying to claim all the code as theirs stoopid Oracle antics. (you know, like what happened to *.gif)

0
0
0.000
avatar

I always thought Java and JavaScript were the same things...so much for being a man of knowledge.

It is really cool to know that Hive has developers everywhere waiting to tweak things around. I tried dipping my foot into python for about a week, it was like learning to swim in the deep part of the pool.

0
0
0.000
avatar

Haha yeah and Python is the easy one.

😅

0
0
0.000
avatar

Lol...I need to find more time for it.

0
0
0.000
avatar

"Sequel" is for the microsoft folks. MySQL is not microsoft so My-S-Q-L is the correct use. And I'm hungarian and we pronounce abbreviations by the letters in most cases so "my-ash-coo-ell" for us (for the whole hungarian developer community, which is about 10 people, so we dominate the world).

Posted Using LeoFinance Beta

0
0
0.000
avatar

I had no idea that Java was so ubiquitous.
I wonder why it isn’t recommended as a top language to learn?
I always see Python and JavaScript.
8CBD73EE1ACC4FE196185478D7B42491.jpeg

Posted Using LeoFinance Beta

0
0
0.000
avatar

Scripting languages are faster to code and easier to maintain.

Javascript is especially useful because it can talk to both websites (HTML) and make server calls easily while not blocking the code waiting for a response.

Basically the only reason to learn Java over scripting is that it's faster execution and used for a bunch of random stuff.

0
0
0.000