Monte Carlo simulations (or methods) is the technique of applying randomness and the Law of large numbers to the solution of various scientific and engineering problems. One of its first documented uses was by Stanislaw Ulam and John von Neumann for nuclear weapon simulations after WWII [1]. In this post I want to provide examples of some simple uses of Monte Carlo simulations. We'll start with the classical example of calculating the value of \pi by throwing darts. Estimating pi Suppose we take a square board and inscribe a quarter of a circle into it. We then proceed to throw darts at the…
Invoke “groupby” with two categorical columns in Python Pandas, and get a two-part multi-index: Turn into a data frame with unstack: The post Pandas groupby with two columns: Reshape results with unstack appeared first on LernerPython.
A “groupby” call in Python Pandas is normally sorted by index. But if you’re grouping by month name, April will come before January. Pass “sort=False”, and the index will reflect […] The post Pandas groupby with sort=False: Keep groups in order of appearance appeared first on LernerPython.
The simplest grouping in Python Pandas is groupby: For example: Returns a series whose index is the unique values from passenger_count. The post Pandas groupby basics: Aggregating a numeric column by category appeared first on LernerPython.
My team develops a microservices application on Kubernetes, with hundreds of PRs opened each day. To let engineers test and review those changes in isolation before they’re merged, we give every pull request its own ephemeral environment. We use Pulumi to define those short-lived PR environments from a component resource that’s shared with our long-lived Dev, Stage, Prod environments. Each PR gets its own Pulumi stack and Kubernetes namespace, which we tear down once the PR is merged or closed. In this post, I’ll walk through how we’ve implemented this pattern and what we’ve learned from…
You can remove NaN from a Python Pandas data frame with dropna, but be careful: It removes rows with even one NaN, which can be overkill. Pass “thresh” to allow […] The post Pandas dropna with thresh: Drop only rows with too many NaN values appeared first on LernerPython.
Coming to Python Pandas from NumPy? You’ll reach for np.isnan: Unfortunately, this works. Better, use s.isna (or s.isnull). But the best way to drop NaN? Use dropna: The post Pandas dropna: The best way to remove NaN values from a series appeared first on LernerPython.
I’m a big fan of exercises, which is why the LernerPython platform includes hundreds of them — all using my in-browser practice system, which handles Python, Pandas, and Git, along […] The post New on the LernerPython practice system: A visual debugger appeared first on LernerPython.
PyArrow dtypes in Python Pandas are nullable (with pd.NA): s is: 0 101 2 30dtype: int64[pyarrow] The dtype is int64, but allows nulls. (Use np.nan? It’s turned into pd.NA.) The post PyArrow dtypes in pandas: Nullable integers with pd.NA appeared first on LernerPython.
You have a Python Pandas series with ints + NaN. You don’t want float forced on you. Solution: Use the “extension” type Int64 (note Initial Caps) and pd.NA: s is: […] The post Pandas nullable integers: Keeping ints with Int64 and pd.NA appeared first on LernerPython.
Python Workers allow developers to run Python web frameworks and AI orchestration libraries natively in the Cloudflare Workers runtime. You can seamlessly integrate with Cloudflare's ecosystem including D1, R2, and Workers AI without writing any JavaScript glue code.
Some of the most satisfying work I do happens one on one. You arrive with a real problem from your real job — code that will not behave, an architecture […] The post Get Python help, one on one: my coaching sessions appeared first on LernerPython.
Some of the most satisfying work I do happens one on one. You arrive with a real problem from your real job — code that will not behave, an architecture […] The post Get Python help, one on one: my coaching sessions appeared first on LernerPython.
Missing data? NumPy calls it nan. Python Pandas displays it as NaN. But: Pandas doesn’t define pd.nan or pd.NaN. NumPy removed np.NaN in version 2.0. So you have to refer […] The post NaN in Pandas: Why you need np.nan, not pd.NaN appeared first on LernerPython.
Missing data in Python Pandas? We use nan (“not a number”), which comes from NumPy. np.nan is a float, but not a normal one: The post np.nan in Pandas: Why missing values break comparisons appeared first on LernerPython.
What is a “callable” in Python? Typically, a function or class. But really, it’s anything with __call__ defined: The “callable” builtin basically returns True if it finds __call__. The post Python callables: Any object that defines __call__ appeared first on LernerPython.
If you invoke +=, Python can use __add__. MyClass implements __add__ (calling print for debugging): m1 now refers to a new object, and its repr is: MyClass instance, vars(self)={‘x’: 30} The post Python += calls __add__ and rebinds to a new object appeared first on LernerPython.
How does the “in” operator work in Python? – If an object defines __contains__, then its (boolean) result is returned (coerced to bool).– If not, then Python iterates over it […] The post Python in operator: How __contains__ speeds up membership tests appeared first on LernerPython.
I really enjoy reading e-mail newsletters. They’re clever, informative, and funny, and provide me with lots of food for thought — as well as professional information that is crucial to […] The post Newsprint: Turning e-mail newsletters into a personal PDF appeared first on LernerPython.
In Python, we use “or” for conditions. | is bitwise (not boolean) “or”: x | y # 15, or 0b1111 | runs __or__. On dicts, | combines. The post Python | operator: Bitwise or and dict merging with __or__ appeared first on LernerPython.
The % in Python, on numbers, is modulo: But str uses __mod__ for interpolation: Same operator, same magic method — but totally different. The post Python `__mod__`: Modulo for numbers, interpolation for str appeared first on LernerPython.
Just as + in Python invokes __add__, other operators invoke other magic methods: • – is __sub__• * is __mul__• / is __truediv__• // is __floordiv__• % is __mod__• ** is __pow__ Your methods can […] The post Python operator overloading: The arithmetic magic methods appeared first on LernerPython.
I've been using Claude Code several hours a day for months, and I'm having a blast. But an agent does what you tell it, not what you meant — which is why validating your results now matters more than the results themselves. The post Claude Code always produces something. That’s the hard part. appeared first on LernerPython.
If you tell Python it’ll run x.__add__(5), which handles the int 5. But what about int.__add__ doesn’t know how to handle x. So it returns NotImplemented — and Python turns […] The post Python’s __radd__: How reflected operators handle 5 + x appeared first on LernerPython.
If your Python class implements __add__, it should usually return a new instance of your class, not an int: The post Python `__add__`: Return a new instance, not an int appeared first on LernerPython.
How can your Python object support +? Implement __add__: The post Python operator overloading: Implementing __add__ for + appeared first on LernerPython.
Operators in Python are turned into “magic” method calls: x + y # becomes x.__add__(y) So: Different errors from different methods! The post Python operator overloading: Why x + y and y + x give different errors appeared first on LernerPython.
Because Python functions’ defaults are kept in __defaults__, avoid mutable defaults: add1.__defaults__ # ([1, 1, 1],) The post Mutable default arguments in Python: why `__defaults__` bites appeared first on LernerPython.
When a Python generator yields, its stack frame remains — inspect it for the current line and local variables: The post Python generator frames: Inspecting locals with gi_frame appeared first on LernerPython.
Summary: If you get Python crashing immediately in BirdNetPi, try uninstalling the pyarrow Python package. I’ve got a BirdNetPi set up at home. This is a bit of software that runs on a Raspberry Pi and listens on a microphone (I’ve mounted mine on the outside of an upstairs window, using a 3D printed mount/cover […]
As someone who teaches Python programming for a living, I’ve spent the last few years wrestling with the educational implications of AI. I’m changing everything I do to adjust to […] The post MIT just called for an educational revolution appeared first on LernerPython.
July and August are often when people take a break or go on vacation. But for me, this summer has been super busy, full of writing and improving LernerPython based […] The post AI seminars — just one of the big changes at LernerPython appeared first on LernerPython.
pythonpandasexcerpt only · body stays at the source
There has been a long gap between posts (again!), and a lot of the stuff that I’m going to talk about in this post is fairly old too. Oh well, life keeps getting in the way of blogging. Anyway, I’ve recently submitted a talk for the FOSS4G UK 2026 conference in Leeds in October, and […]
Using PyArrow dtypes in Python Pandas isn’t always faster: %timeit s_pyarr.mean() # 21.5ms%timeit s_np.mean() # 47.3ms %timeit s_pyarr.nlargest(10) # 667ms%timeit s_np.nlargest(10) # 663ms The post PyArrow dtypes in Pandas aren’t always faster than NumPy appeared first on LernerPython.
When we build agents, we often want to give them the ability to browse the web: open webpages, navigate from one page to the other, and read the content of a webpage. By combining Pydantic AI with the Playwright capability from Pydantic AI Harness, we can build agents that browse the web safely and programmatically. Using Pydantic AI with Microsoft Foundry models Pydantic AI is an open-source model-agnostic framework from Pydantic for building LLM-based applications and agents. It's type-safe and supports OpenTelemetry, making it a great choice for robust production applications. We can use…
PyArrow strings in Python Pandas are smaller than Python strings. But they’re also far faster: %timeit s_pyarr.str.len() # 106 µs%timeit s_py.str.len() # 1.6 ms In many examples, PyArrow was far […] The post PyArrow strings in pandas: Faster and smaller than object dtype appeared first on LernerPython.
Want PyArrow dtypes in your Python Pandas data frame? The dtypes are double[pyarrow], int64[pyarrow], and string[pyarrow], not the normal NumPy ones. Note: This is still experimental… but it’s also the […] The post PyArrow dtypes in pandas: Reading CSVs with dtype_backend appeared first on LernerPython.
Using Python Pandas 3? Strings use PyArrow, not Pandas 2’s Python strings (dtype “object”): The post Pandas 3 string dtype: PyArrow strings cut memory use appeared first on LernerPython.
Model Context Protocol (MCP) is an open protocol that describes how agents can connect to external tools and data sources, and is now widely supported by the most popular coding agents (like GitHub Copilot, Claude Code, and Codex) and agent frameworks (like LangChain and Pydantic AI). If you want to give agents a standard way to access the data in a database, you can build your own MCP server and expose tools for the agent to query or even modify data. But you need to design your MCP server carefully, to ensure that agents can do everything that users want - but nothing that you don't want…
If you want to get better at Pandas, the hard part isn’t finding tutorials. It’s finding problems worth solving. Most exercises hand you a tidy little table of five rows […] The post Free real-world Pandas exercises, with solutions appeared first on LernerPython.
pythonpandasexcerpt only · body stays at the source
Want rows in a Python Pandas data frame that might have several values? You can use the | operator, but be sure to use () to avoid precedence issues: The post Pandas OR conditions: Filtering rows with | and parentheses appeared first on LernerPython.
Stacking loc to filter a Python Pandas dataframe? Order can matter: The post Pandas loc filter order: Put the most selective filter first appeared first on LernerPython.
Filtering rows in a Python Pandas data frame? Use .loc: pd.col refers to the previous line’s returned data frame. So we can stack them: The post Filtering rows in Pandas with .loc and pd.col appeared first on LernerPython.
How much memory does a Python list use? sys.getsizeof will tell you, sort of: It reports the memory used by the list, but not its elements: The post Python list memory: Why sys.getsizeof ignores the elements appeared first on LernerPython.
Want stderr in your Python program to go somewhere else? Use redirect_stderr in contextlib: The post Redirecting stderr in Python: Using contextlib.redirect_stderr appeared first on LernerPython.
Writing a bunch to non-stdout in Python? You might want to use redirect_stdout from contextlib: The post redirect_stdout in Python: Sending print output to a file appeared first on LernerPython.
Want to print to stderr, not stdout, in a Python program? Use sys.stderr and the “file” kwarg: stdout and stderr often look identical in a terminal — but they aren’t! The post Python stderr: Printing with sys.stderr and the file kwarg appeared first on LernerPython.
Normally, “print” in Python writes to stdout. Write elsewhere with the “file” keyword argument: The post Python print to a file: Using the file keyword argument appeared first on LernerPython.
Want stdout from your Python program to go elsewhere? Assign sys.stdout to a writeable file. (Don’t forget to keep the original around!) Better: print’s file kwarg. The post Redirecting sys.stdout: Assigning print output to a file appeared first on LernerPython.
Optional Google Analytics helps us understand visits. Microsoft Clarity records masked interactions to improve the site. Optional tools stay off unless you choose them. Privacy details.