How that goal is achieved

How that goal is achieved

Level 2, the second data engineer

The “level” refers to the number of humans working on data engineering:

2 people – less pain for ourselves. It’s necessary to start packaging, documenting, automating, and all those cool things to work within a team. There is still no slack, but from time to time, it makes sense to refactor an odd choice made earlier. All the work is still around making data available.

Much is already covered in this whole post, but I will add some additional notes.

There is an immediate and substantial communication overhead from one to two. It’s like going from living alone to living with a partner – many things need to be explained and agreed upon. On the other hand, more stuff will get done, double the experience to develop better solutions, etc.

One aspect of communication is that ownership and responsibility become essential topics. Not that who owns what, but how do you get from one person owning everything to two people sharing and caring? One possible approach is to plan and hire in such a way that an essential piece of the platform will be the primary responsibility of the second person. That way, they will naturally become the owner of it. Of course, there are other ways, but the main point is that it should be an open discussion before the second person starts, ideally even before the job ad is written.

As I’m hinting here, planning is becoming more critical as time goes on and the environment changes. By the time a company goes from zero to two data engineers, the number of analysts, product people, revenue, and many other things have probably multiplied by a lot. That usually also means that the hectic early changes will become less frequent, and planning will begin to make sense.

The following article will be about the “why” part, the vision statement.