What I would do differently getting into Data Engineering [in 2024]
![What I would do differently getting into Data Engineering [in 2024]](/content/images/size/w960/2024/01/Screenshot-2024-01-09-at-09.29.46.png)
It's been close to 10 years since I started my Data Engineering journey. Throughout my career, I have had the opportunity to work with startups, enterprises, and consulting firms. I started out as an individual contributor and then recently moved into a management role. Over the years, I have worked with both legacy and modern data stacks - perhaps not all, but I have seen most of what the industry has to offer.
As the field of data continues to grow and becomes more popular every year, I have decided to put together a list of things I would do differently if I were to start my career in 2024. Hopefully, folks who are early in their journey will find this helpful.
These are in no particular order.
It's All SQL
Firstly, a lesson from my journey is the importance of embracing SQL wholeheartedly. It might be an obvious one, but having originated from a software engineering background, I initially neglected the power of SQL. However, with time, I've come to recognize it as an indispensable skill with exponential returns. You will always need SQL and the best thing about it is that it's all the same in all environments!
Sure, some databases support different dialects, but the core concepts remain the same.
Hence, my advice to newcomers is to delve deep into SQL early on; its mastery is an investment that will undoubtedly pay off manifold in the long run.
My current favorite resources for learning SQL are:
- The Mode SQL Tutorial
- Codeacademy SQL course
SQL is merely one of many tools available to data engineers. To learn how to use it effectively and delve deeper into the subject at large, you can consult my recommended reading list below.
Daniel Palma
Actually, It's All People
The revelation that took me years to internalize is the pivotal role of social interactions and network building. Contrary to the pervasive notion of the "antisocial" developer, I've learned that success in the data domain, like any other industry, is profoundly influenced by human connections.
Building a network early on, making friends within the industry, and understanding that people, fundamentally, are willing to collaborate and assist, can significantly impact one's trajectory. Talk to your coworkers, reach out to your peers, and comment on LinkedIn – communication is just another skill that we can improve by doing it. All those
Recognizing that collaboration and relationships are vital components of career progression has become a cornerstone of my approach and if I could go back in time, I would start as soon as I could.
Feedback as a Catalyst
Another crucial aspect I would revise is my approach towards feedback. While I've become adept at seeking feedback over the years, I now realize its significance from the outset of one's career. Actively seeking feedback from peers, managers, leadership, and even clients provides invaluable insights. Do not settle for your mandated quarterly/yearly feedback sessions – during those periods folks are under a lot of time pressure and might not be able to dedicate enough mental space to collect their thoughts about you. Instead, ask for targeted feedback about smaller deliverables you produce or actions you take.
As a practical pointer, approaching individuals promptly after completing a task ensures that their feedback is based on fresh memories, resulting in more constructive and actionable advice. I routinely ask participants in a call we just left about their thoughts regarding my performance.
As an exercise, next time you present something to a group of people, prepare a simple feedback form that you send out to them after the call, I guarantee you will see some great, actionable thoughts!
The Art of Saying No
Finally, a skill I would prioritize is the ability to say no. Early in my career, I often found it challenging to decline additional tasks or projects, fearing it might hinder my professional growth. In reality, saying no is one of the biggest enablers of quality focus time.
However, with experience, I've learned that judiciously saying no is an essential skill. It allows for a focused and efficient use of time and energy on tasks that align with personal and professional goals. This is one of those easier-said-than-done things, as even now, sometimes I find myself thinking "Why the hell did I just say yes to that", but the important part is recognizing these mistakes and learning from them.
In conclusion, these reflections encapsulate some of the lessons I wish I had known when starting my data career. To those embarking on this journey in 2024, I urge you to embrace SQL, actively seek feedback, prioritize social interactions and network building, and learn the art of saying no when necessary.
If you are looking for help with navigating the industry, feel free to reach out to me – I'll do my best to answer!
Member discussion