Whenever I have to deal with anything “data science” or “data engineering” or what the fuck we call it this week, maybe it is all (sarcastic Homer Simpson voice) AI, the whole notebook stuff drives me nuts.

Like I think they are fun for some stuff, but it is so weird for a lot of use cases at work. Mostly because we have all of these great developer tools over here, and then most of them aren’t really usable within the notebooks.

Is this a problem because I’m more of a “software engineer” (I refuse to use the engineer title myself) than a “data engineer”, yes. But here is the reality of doing software in any company everything has to talk to each other in some way.

Development, operations, “data” all has to do work on the others work sometimes. Or a lot of the time. Or you have to take on the role of some of the others for a while.

When I see all of the code blocks in the notebooks I just think: can I have these in a git repo as files that are linked into the main document when you work on it as a notebook? So that I can write the code in PyCharm or VS Code or whatever?

I like literate programming by the way. But I think there is a place for it and a place where it isn’t a good fit. But since everything is based on it at this point, it should be destructed into a way where it is less of a pain for different disciplines to deal with.