FRESH

Hacker News

Home

An oral history of Bank Python (2021)

173 points by tosh

by alexpotato

0 subcomment

I've worked at a bank and several large hedge funds.
Some additional interesting tech stories I would add:
- in 2010, the bank had retail Good Till Cancel orders from 1997. I think one was "Buy INTC at $6"
- There is a mix of "I didn't know technology could do this" in the good sense and "I'm amazed this code a. works at all and b. hasn't had an outage in 6 years"
- There is a strong desire, I chose this word carefully, to migrate off of legacy systems. That being said there are several; big issues: 1. it's a GIGANTIC amount of effort with often unclear ROI to the business, 2. upside is capped (maybe you get a promotion) but downside risk is huge (you could tank the business with an outage). 3. Slow, gradual refactors are generally better here but some things can only be "big bang" for various reasons
- You tend to see old but performant and battle tested systems get retired in favor of shiny, new systems with lots of bugs. Why? It looks better on a resume to say "I retired old, crufty legacy system and rolled out a new system" instead of "I refactored old system to be better"
- The complexities are wild e.g. Korean trading requires: a. traders to be licensed in Korea (even if they are working in NYC), b. servers to be in Korea c. tagging orders with not only their executions but also the exchange rate at the time
- There are entire SYSTEMS built to track trade breaks (e.g. Bank A doesn't agreed with Bank B on fill 1248383). Some of these trade breaks are open for for YEARS due to litigation, companies going out of business etc
I could go on and on about this.
If anyone is ever interested in having me on a podcast to talk more about it, I would totally be up for it.

by elemeno

3 subcomments

To the best of my knowledge much of this originated with SecDB/Slang at Goldman - SecDB (securities db I believe) being the object store and slang the somewhat quirky C like language that ran with it (also the only language I’ve used professionally that let you have spaces in the variable names).
Some of the folk that built that (or worked on it) ended up at JPM and Merrill where they built the Python centric version - Alpha and Quartz respectively. Barclays Capital has/had a similar system as well I think, but it’s not one I know about offhand - they did though, memorably, have a system that was pretty much Haskell-in-Excel.

by tsukikage

3 subcomments

When first encountering these ecosystems and looking at the various pieces they contain, one may repeatedly ask: "why didn't they just use <off-the-shelf solution> for this problem instead of writing this component/subsystem from scratch"?
The answer is often that the battle-hardened mature off-the-shelf solution did not exist at the time the code was written. You're doing software archaeology.

by axus

1 subcomments

What a well-written account of "how things are done".
> Time to drop a bit of a bombshell: the [Barbara] source code is in Barbara too, not on disk. Remain composed. It's kept in a special Barbara ring called sourcecode.

by amyjess

0 subcomment

I remember reading this article when it was first posted here five years ago, and I've been fascinated by Bank Python since. It actually reminds me of a number of systems I've come up with in my head but never told anyone about or wrote down in any way.

by manoDev

0 subcomment

If you ever worked with mainframe, you'll see a lot of similarities:
- Unified interface for object stores
- Source code stored with data files
- Job runner
I also see some similarities with Lisp machine, the fact Python also has a REPL, and able to dump/restore image state (but in this case discrete objects are serialized, not the entire memory).
This might sound crazy for people used to having 90% glue code / 10% business codebases, but to me seems like a very efficient way to have users directly drive what is effectively a large computer, and more like how things used to be.
The drawback is that it seems to be a monolith, and maybe hard to reimplement on top of more modern foundations. But as a general API, it seems to make sense.

by TZubiri

2 subcomments

>Applications also commonly store their internal state in Barbara - writing dataclasses straight in and out with only very simple locking and transactions (if any).
Right out of the gates, it's crazy how this contrasts with Mercury's Haskell infra
https://blog.haskell.org/a-couple-million-lines-of-haskell/

by skissane

4 subcomments

I think it is a pity they’ll likely never open source any of this stuff
Of course, financial institutions have a lot of “secret sauce” - such as financial models - you’d never expect them to release.
But this kind of underlying infrastructure isn’t really “secret sauce”

by mhh__

0 subcomment

People turning up in hedge funds (i.e. much smaller) and trying to rewrite the bit of a bank they used to work in's equivalent of this article is so annoying.

by horticulturist

2 subcomments

> This is because clients generally do not ring up about pennies.
I’ve had clients ring up about pennies… it can be crazy what some people are motivated by

by piinbinary

0 subcomment

Prior discussion: https://news.ycombinator.com/item?id=29104047

by valzam

0 subcomment

> One of the great drawbacks of "Cloud Native Computing" as it now exists is that it's really, really complicated. It is often more complicated than the old, non-cloud, sort of computing. In order to deploy your app outside of Minerva you now need to know something about k8s, or Cloud Formation, or Terraform.
Highly agree with this. I think it's very underappreciated in startups that if you want people to deploy a lot of small services you have to make that really super easy. I always thought that the value of things like Spark is that you can run "things" without having to worry about how they run. K8s is similar but much more complex. AWS Lambda is nice but also comes with a lot of baggage at scale. I always wanted to try something like Dapr, which seems to provide a very opinionated happy path for application development.

by Havoc

4 subcomments

I've seen similar inside large financial orgs - what struck me was how there are these huge amounts of people that spend their entire working life inside this alternate IT reality. It's not unlike SAP consultants where their skillset is tied to one company.
Also...these things tend to have fuckin terrible documentation. Good luck figuring any of this out. And you can't google it and your AI is just as lost as you

by nxobject

0 subcomment

I would give my left leg to learn how the permissions system worked – do end users (and PHBs) get to edit the rules directly? I fully expect some HR ass to go:
```
  can_view(Person) :- didPITAOnlineTraining(Person), ...
```

by coredog64

0 subcomment

And I thought rewriting 3rd party packages to work with AFS was crazy

by tomrod

0 subcomment

A few of the scientific computing companies from the early 2010s got their traction due to Dodd Frank-required scenario and stress testing. SAS was not up to the challenge and R could not multithread.
These initiatives were independent of Minerva and Athena, which was good but not very useful to the more mundane parts of the bank everyone off the financial trading floors care about.

by frays

3 subcomments

The previous discussion was fascinating: https://news.ycombinator.com/item?id=29104047
Does anyone working at one of these banks or similar know if this information still holds true?
And have any of the banks started using uv yet? Or will they forever be using pip?

by roywiggins

0 subcomment

Weirdly not dissimilar from MUMPS systems.

by jgalt212

3 subcomments

> but the default ring is more or less a single, global, object database for the entire bank.
Is this really the case? I'm sure there are plenty of transactions that for umpteen different reasons must not be exposed on a global level.

by debamitro

0 subcomment

Exhilerating article, but this has been shared on HN countless times

by wirthjason

0 subcomment

```
  s/barbara/sandra/g
```

by matt_daemon

0 subcomment

This is one of my fav blog posts

by Ozzie-D

0 subcomment

[flagged]

by mv_d5339e31

0 subcomment

[dead]