Hisashiburi! My Journey to Data Engineering

It's been 1,006 days since the last post on this blog. To say that a lot has happened since the last post would be the biggest understatement of my life.

To keep it short and efficient, here's the bullet list of milestones in the last 3 years:

1. After living and working in Singapore for 7+ years, I moved to Ireland. 
2. I've been living in the "Emerald Isle" for almost 2 years now (September 2020 marks my 2nd year of officially relocating here)
3. I've completed a short diploma course on Python programming from a school here in Dublin (which proved to be of absolutely zero, nada, zilch use to me in my career move, but more on that further in this post)
4. I'm no longer in Customer Service and now in Data Engineering, which means....
5. I've finally found what I want to do in life 😃 at least for the time being, until the next "adventure"

And that would be the focus of this post: how I found what I want to do in life and landed the related job. Hope this is something useful to someone out there.

Totally unnecessary context (skip this part if not interested): I went from language studies, translation, didactics of languages. It was a constant exploring or simply an "excessive and aimless wandering" of person without direction in life. 

That not so long winding road landed me in customer service. You must know though that job was not my "shining moment". Let's be honest: What introvert would happily claim that talking to strangers is something he/she particularly finds thrilling, let alone being the outlet of their complaints (and I assume their family's complaints, their neighbors' and their neighbor's family's complaints as well)? 

If I could statistically describe what quality time spent with loved ones is, then the chart below would be somewhat accurate: 

• ~20% actual talking to each other and 
• ~80% being together as each one minds his/her own business, in quiet solitude

And by "loved ones" I refer to people I profess to adoring to bits and pieces and would sell any and/or all of my internal organs for! It's bursts of joking, suddenly belting out concert songs scattered in the vast periods of quietness. I enjoy letting people talk so I can listen and respond (if I have to). 

Note to strangers: If I am too chatty voluntarily, it's mostly defense mechanism for fear of socially offending someone. On the flip side, however, if our interactions are mostly stretches of time wherein I'm in dead pan silence while you're in "blabbering non-stop till you drop" mode without it being awkward, then you most likely have first dibs on one of my kidneys or part of my liver, and everything else on my person, a.k.a. I wub you to the moon and back!

Anyway, we're going off tangent. The point is that that I don't have the natural disposition of wanting to be always talking to people and listening to them vent when they don't have first dibs on any of my biological parts. 
It's more likely that just like any typical Asian / Filipina, I had to be pragmatic and so chose the path that allowed me to earn as much as I can to more easily give back to my family back home in the Philippines. And thus came to be that a good decade of my existence, was spent working as customer service specialist.

And the reason for this post is the happy news that I got out of customer service and into data engineering this 2020. Actually, the official title is Data Analyst which on paper seems to point out to front-end stuff; mainly visualization (yes, charts and graphs!), creating web app dashboards, and reporting.

However, almost all my tasks, I've discovered, are always creating data pipelines, creating/maintaining data base tables, lots of Python, SQL - and all the flavors we have (vanilla, Hive QL, Presto), the odd R now and then, creating DDL schemas, a bit of statistics K-means, K-medoids, etc, working or wrestling - whichever you prefer - with Github, and tons of lurking in Stackoverflow (of course! any self-respecting programmer will say that's the "secret sauce"). In summary, the "behind the scenes" / back-end stuff. To take the analogy of a watch, it's those tiny gears that move the dials or the circuitry that drive the data displayed on the screen you see on your wrist that I take care of. "Engineering" bits and bobs.

So how do you score the job and make the career change?

• Do your part and develop the necessary skills. 
• The best way to ruin your life is to believe you're entitled. That the job offer will be served to you on a silver platter with icing and cherry on top has a probability value close to zero. Or if it does get served to you, it's most likely an aberration to most if not all laws of nature, meaning you're probably from an alien species and belong in outer space anyway. Or that you've got connections in high places. One of the lucky ones!
• You get only as much as you put in. I can only speak from what I've done personally. I actively sought means and invested the time to learn how to code. I remember hoarding the living room TV to watch instructional videos on how to code much to my family's chagrin as it meant they couldn't watch their favorite shows. That was during Christmas break in 2017, when I decided to get serious in learning to code, and in one of the few days I get off work. I was doing exercises on my mobile phone while we were on the road, traveling to family reunions. There was one time I was working on a Django web app following a tutorial book on a weekend last year 2019, and it meant skipping my 1st St. Patrick's Day in Ireland in exchange for getting closer to accomplishing the goal of moving jobs. Priorities, gurl. There's only a finite number of hours in a day and something's gotta give. *It was probably a stroke of luck that I chose Python as the first programming language to learn. I just thought its syntax is prettier than Java's or C++. Turns out, it's actually "the" language to learn if you seriously want to get into data. To be fair, you can use any language but the vast majority of powerful libraries for data wrangling were developed first for Python. There's R of course for hard core statistics and Julia as well, but Python is a staple for venturing into data - you don't drive without a car, or get a car out without the wheels after all.

• Admitting you know nothing is the beginning of real learning
• Rewire your brain or thought process. To be completely honest, the pivotal moment where things just finally "clicked in my brain" in terms of learning to code was when I had to do a project that had real-life impact. Not from a text book or from class. The project was work related: automating case routing in Salesforce. Thinking of how to solve that one problem trained my brain to think logically. I had to learn how to "pseudo-code" first, how to break a vastly bigger concept into the simplest parts, then code those parts and put them together to build the whole, working solution.
• Be willing to be Jon Snow ("You know nothing!") and have the courage to still dive in, head first into the unknown. Another challenge for that project was that I couldn't do just a simple "pip install <insert package name here>" in the terminal. My scripts were throwing errors. Even if I still vaguely thought "functions" worked like magic and knew nothing at all, I tinkered with a python library called "simple salesforce" (API that allowed manipulating Salesforce objects). The root cause of the error? It was that the URI for our company's Salesforce instance didn't match the default one on the package. I almost had a major brain meltdown but in the end, I managed to reverse engineer the API and in the process, gained a foundational knowledge that transcends all programming languages, not just limited to Python: HTTP protocol implementation. Also it bagged me a recognition award from work (it was working!!!) and most importantly, it was another item to add on to my CV as part of my coding portfolio.

• Continue to improve those skills as you build a portfolio
• A certificate does not a programmer make. Though it's true that I chose to get a diploma in Python programming from CCT college of Dublin, that was not how I really learned to code. I believe to this day, that I learned more from a 10-dollar (USD) course in Udemy about programming  and just finding more things to automate or build, to practice coding over the last 2 years or so, rather than sitting in those night classes after my day job.
• Collect a repertoire of projects that solve real-life problems.
• List all those projects in your CV and present them in such a way that matches closely the job requirements in the posting. You have higher chances of getting hired for a job where your prospective team mates and manager can see that you already do or know how to do the tasks involved in the job: "...you're the team member that they haven't just hired yet" ~Python Programmer Youtube Channel.
• If the job is a technical one, an exam might be inevitable. But you can prepare for it by practicing; applying what you know in your every day life. In my case, I had to do a time-limited online exam with 2 parts, first Python and then SQL, each with 4 sub-modules of about 10 questions. It was the most grueling 30 minutes of my life and I only got through to the 1st sub-module of the SQL part. And boy was I glad to have put in those hours of practicing in Jupyter notebooks: I could've resorted to using normal means to create reports that I had to submit for my then current job but I'm glad I chose to transform them into  practice opportunities. The recruiter revealed to me that I did well on the Python part (pandas library); my score was one of the highest and so it didn't matter that I didn't finish the exam.
• Practice. Code, code, code. Having practiced means you get almost second nature knowledge. The more significant part of the interview process were potential team mates peer-reviewing my code - my portfolio. They asked questions like "what did you use to store the values - list or dictionary - in this section here of this script? why?". If you didn't code those lines yourself, you'd have to go the "fake it till you make it" route. All I can say is good luck to you. You might have more chances of finding water in Mars, while still stuck in earth, than making it alive out of that situation with your dignity and integrity still intact.
• A roll of tissue could sometimes be more valuable than that certificate you cherish. They never asked to see any certificates from me. I even had to volunteer to the talent recruiter to submit mine (yep, that Diploma I got in Dublin) and to which she replied "You don't need to submit that..." with an undeniably puzzled tone, as if I had suggested the most horrendous of ideas like sending over my baby photo album.

• Check for job postings and determine which one you're sufficiently interested in. 
• Do your research, keep your eyes peeled for job openings. This circles back to the "entitlement point". Opportunities don't really come waltzing your way. Sometimes they knock on your door, but you still have to make the effort to actually stand up and open it. Also, aside from external job sites, why not watch out for postings in your current company? There's good reason why you stuck with them thus far and they've probably taken care of you quite nicely; maybe you can continue to be of service to them, but in a different role. Continue the lovely working relationship.
• There's no such thing as a perfect job. I would've preferred to do only web or desktop apps, in Python only, and never touch any Microsoft application for the rest of my life on earth. But this is the real world where there are only real companies willing to take you on if you're equally willing to help them solve their real world problems with whatever tool or app is available. So when they ask me create a SharePoint list that gets automatically synced with a .csv file from a server using Microsoft Flow and Excel, then I get down to working. Even if I absolutely abhor Excel and would rather give up my first born child (that I probably won't ever be pregnant with, woah, immaculate conception!??!?) rather than work with a nasty bit of vlookup, I move on and get on doing stuff.

• While waiting for that elusive opportunity, do your utmost best in your current job.
• All job experience is still experience. Even if you feel you're not cut out for the current job you're in, just suck it in and give it your best shot still. The natural disinclination does put you at a disadvantage because your brain just refuses to learn. But you just have to grit your teeth and work hard despite and in spite of things. Surprisingly, my decade long stint in operations (customer service) actually meant I understood the business side of things "the why" in those technical spec requirements that engineers or BI developers don't necessarily get. For someone who prefers being left alone but was doing a job that required being on the call for people needing help; it really did feel like  pulling tooth out every day, all those 8+ years! Imagine, It took me maybe 4 training sessions for eStore - one of the many work streams in my previous job - and a good 3+ years before I could handle a customer call / case. And I felt my coworkers thought of me a stupid bimbo, they had every right to do so.
• What you lack in process expertise (or just plain interest) make up for it with an impeccable work ethic. It's a way to show gratitude you have a job that pays for food on the table, roof on your head, and clothes on your back - and your family (for Asian / Filipinos like me hehehe). Make yourself easy to work with and a delight to your direct reporting manager and you'll find out it's not hard to get those references for your CV. Things can be worked out with a willing team player and someone who's a joy a to work with will almost always be preferred. Even if you're this genius super-talented person but "does not spark joy" in the team, you're most likely be booted out the way Mari Kondo doesn't even bat an eye lid when discarding things to avoid clutter

By the way, the thing that "sparks the most joy" for me in the new job is that, setting aside the 15-minute "daily stand ups" and other short necessary meetings, I get to spend the entirety of each working day staring intensely at my computer screen(s), without as much as flinching or blinking. To the point one would think I was brain-dead or a zombie, the only indication of a still cognizant nature being the frantic typing noise on the keyboard and the occasional frustrated exclamations of "Why does that throw an error?!!" or "Arrrgghh!! Can't catch that bug!!"

 Ah...heaven!