This is a classic piece, and I love the contradictions in the text. It encapsulates my feelings on good software and code that it almost becomes an art than a science.

Thanks for sharing. We use all pytest-style tests using pytest fixtures. I’ll keep my eyes open for memory issues when we test upgrading to python 3.12+.

Very helpful info!

I’m most excited about the new REPL. I’m going to push for 3.13 upgrade as soon as we can (hipefully early next year). I’ve messed around with rc1 and the REPL is great.

Do you know why pytest was taking up so much RAM? We are also on 3.11 and I’m probably going to wait until 3.13 is useable for us.

EOL for 3.8 is coming up in a few short weeks!

So cool!! Mercury is definitely the most mysterious inner planet due to its difficulty to get a space probe there even though it’s the closest planet.

The spacecraft will arrive next year, and I can’t wait for all the Science it will uncover!

TIL this exists

I also like the POSIX “seconds since 1970” standard, but I feel that should only be used in RAM when performing operations (time differences in timers etc.). It irks me when it’s used for serialising to text/JSON/XML/CSV.

I’ve seen bugs where programmers tried to represent date in epoch time in seconds or milliseconds in json. So something like “pay date” would be presented by a timestamp, and would get off-by-one errors because whatever time library the programmer was using would do time zone conversions on a timestamp then truncate the date portion.

If the programmer used ISO 8601 style formatting, I don’t think they would have included the timepart and the bug could have been avoided.

Use dates when you need dates and timestamps when you need timestamps!

Do you use it? When?

Parquet is really used for big data batch data processing. It’s columnar-based file format and is optimized for large, aggregation queries. It’s non-human readable so you need a library like apache arrow to read/write to it.

I would use parquet in the following circumstances (or combination of circumstances):

• The data is very large
• I’m integrating this into an analytical query engine (Presto, etc.)
• I’m transporting data that needs to land in an analytical data warehouse (Snowflake, BigQuery, etc.)
• Consumed by data scientists, machine learning engineers, or other data engineers

Since the data is columnar-based, doing queries like select sum(sales) from revenue is much cheaper and faster if the underlying data is in parquet than csv.

The big advantage of csv is that it’s more portable. csv as a data file format has been around forever, so it is used in a lot of places where parquet can’t be used.

Wow everyone seems to love P3 but I actually liked P4 better. I mean I really enjoyed both, but P4 was a more immersive experience for me. I should reboot my vita and play it again.

I really felt like P4 had deeper connections and relationships between the characters. It felt more real, and that made the tension in the game more exciting. I love every second of it and am still trying to find a game like it.

Don’t get me wrong, P3 was great also. The gameplay was superb and the characters were all great. But P4 still has a special place in my heart.

The autocomplete is nice but I don’t find it a game-changer. The comment about writing tests is on point though, but that’s the only place I found out useful.

Python does not follow semver.

https://docs.python.org/3/faq/general.html#how-does-the-python-version-numbering-scheme-work

Nice! I’m a big fan of Iceberg, and it’s nice to see books coming out for it. I used it quite a bit with Spark, and it’s a pleasure to use.

I’m waiting for the python support to be complete, and I can see myself using it full time. Right now, I’m trying to use DuckDB and Python for nearly everything outside of the database, and the only thing missing is good Python support for Iceberg.

I disagree. I think the default option should be what users expect, and users expect “copy” to do exactly that: copy without modifying the text.

While it would be ideal to have all datetime fields in databases and other data stores be time zone aware, that is certainly not the case. Also, SQLite (and probably others) do not have great support for time zones and it’s recommended to store datetimes as UTC (typically unix timestamps).

Deprecating utcnow was a good idea, but they should have replaced it with naive_utcnow. Oh well.

Just because you can get part of your education remotely or through self-learning didn’t mean “anything can be learned online”.

And if you were hiring a math tutor for your kid, would you prefer a self-proclaimed expert from watching YouTube videos or would you want someone who got a degree from a credentialed university? And even if you don’t care, why are you surprised that others would be skeptical of the YouTube expert?

Remote learning can be fine for some things, and self learning through informal channels are also fine, but it’s not a full on replacement for formal education in all cases.

No sorry, that’s just fundamentally false. You can’t just learn titration techniques from watching a video. You can’t learn phlebotomy without an instructor watching you do it to a patient. Hell, you aren’t learning how to drive a car from playing a video game.

And I’m not sure where you are pulling the “if you are that powerful” from. You really have an ax to grind don’t you.

Ah yes, I’m sure the formal training received by doctors, nurses, lawyers, teachers, and engineers is just an over-hyped “education” that can all be replaced by online MOOCs.

There are real problems with education, especially with the costs, but “anything can be learned online” is the worst take I’ve heard in a long while.

I feel the opposite. We should have mandatory voting for all federal general elections. Treat it like jury duty or taxes - voting is a civic duty. You should be compelled to cast a ballot even if you leave it blank because you have no preference.

Of course, this can only workwith automatic voter registration and 100% mail-in ballots.

I’ve said this before to other people, but over time, those tools eventually became what Airflow and other orchestration tools are: defining DAGs and running scripts.

When I was using SSIS, eventually, every task was a C# or PowerShell executor instead of using the built-in functionality. So glad for Airflow and other modern tools today.

I’m shocked.