Fascinating story. I wonder what other dirty secrets state agencies have in stock for us tech stack-wise.
This being said, Excel is most likely the most widespread tool for anything that involves office work. I've seen entire financial algorithms being implemented using Excel alone.
About 15-20 years ago, apparently one of the major ISP in my country was running off excel. Every customer was a row in some file.
ISP agents would come to your house to collect your bill. If someone didn't pay, the agent would flip a cell in their excel row. And that would trigger that person's internet being suspended. Sometimes they would flip the wrong person, and you had to call them, and they would fix it.
Sorry but I just don't buy this. As fas as I know, there was no collaborative editing in Excel 15-20 years ago (no Office 365). The only way for several people to edit the same Excel file stored on a network drive would have been to a) make a local copy of the file, b) do the modifications and then c) overwrite the file on the network drive. This would have destoryed _all_ modifications that occured to the file between a) and c), and would have resulted in such high rate of errors that the company would have been simply impossible to operate.
EDIT: As many people have pointed out, Excel did suppport collaborative editing at the time.
Excel supported collaborative editing of binary excel files on SMB shares under windows NT, back in the 2000s.
Now, "supported" does not mean it was in any way good idea, or that it resembled how collaborative editing works in things like Google sheets or O365. Conflicts, deadlocks, etc were common - it's even why I built my first Rails app - we were given excel sheet and asked to do data entry on it over network from multiple machines, I didn't trust it despite a demo of it working and built a quick and dirty webapp.
You could do it. Excel worked well with querying SharePoint. To most users it would be no different from just editing a file. You could also do it via samba.
(Source: I built one of those during my school holidays but I didn't get paid and I didn't have the balls to ask for payment)
I never said it was one file. I think every agent had one file for their area (a few hundred houses). Then some excel macros to pull all those files into one mega file, that the backend system dealt with.
There are entire trading stations built in Excel, used by multiple traders of one firm, synchronized via network drives, doing requests to the broker using VBA. I saw that around 2003.
About 15 years ago or so, back when I still did web development gigs and the like (because I hadn't accepted yet that the low level system development I loved was what I should do for a job, hard to imagine now), I got requests to build complex or not so complex systems, but with the restriction that "it has to be written in Excel". Most of the time, what they really needed was a CRUD app and a database. Needless to say, I never accepted any of them.
Yes, but from the point of view of an office (lowercase o) worker Access is that thing that, in decreasing order of importance
1) it's not clear what it does
2) it doesn't show them all the data at once all the time
3) it doesn't let them write formulas to work on the data
4) the few that saw Access in action say that they have to call a programmer to do what any Joe can do in Excel
So whatever Access is doing Excel does it better and they can't understand why Microsoft developed a crippled version of Excel to manage tables of data. At this point I almost don't understand it myself anymore despite having seen all the years of Access and Excel ;-)
In a past life the most “fun” I had as a developer on a CAD file processing solution was discovering that engineers loved linking Excel sheets to the CAD files so crucial calculations could be done there. And then having to make this work on Linux servers.
I had to run a very large accounting Excel file inside a web service about ten years ago. The service was born as manually operated Excel sheet plus interaction by email/phone and they wanted to scale it on the web.
I run it on Windows Server and paid somebody to write the Windows service to manage the Excel sheet. I can't remember which language I used for the web service itself.
They eventually gave up, commercial problems, not technical ones.
I'm torn on this. I think many saas type services can be a spreadsheet, but excel is only 98% of the way there. Because the main issue with excel is that it is local only and even then cannot be accessed even by other local programs. Hence all the wacky VBA "scripts" that turn into full flegded programs. I wish excel _would_ become a database, that is, a frontend to a postgres instance that could be run locally or on a server. Then we just set the postgresql srting in our app and everything just works: excel is the frontend, special features run in a sane programming language, can be updated seperately ect.
Since that will nerver happen, someone needs to make an open source speadsheet that runs on postgres. And yes, it does have to be open source because otherwise nobody will learn it. And it needs at least 80% feature parity with excel.
I am well aware of ODBC. Critically you cannot edit the database itself, only copy from it. This makes sense so as not to let some sales guy tank an entire company becuase he misspelled a formula. What I mean is a spreadsheet that literally runs on a database. It could even be sqlite, _anything_ that allows two way communication between the slreadsheet and the application.
spreadsheet gui <--> database <--> application
The spreadsheet does simple data manipulation and acts as a GUI
The database can be mysql, postgres, sqlite (even JSON...)
The application does batch processing, generates documents, sends emails ect. everything excel can't or shouldn't do.
This stack could solve, in my opinion, >50% of all business software.
I have also considered some sort of general data storage but it's too slow. Drop box et al are not made for anything close to real time communication, which is critical for a spreadsheet that needs to talk to a program. Something like 100ms at most. This would also presumably require a perfect .xlsx implementation to read and edit the data which doesn't exist to my knowledge.
Google sheets, maybe excel, can make http post and get requests. It’s trivial to write an api that can read/write to a database on behalf of the spreadsheet.
The trouble with that is it's a central server which means an IT department to fuss with it.
Better would be something like Excel but that works P2P like SyncThing, or even directly using SyncThing itself.
I don't think it would actually be too hard as long as you didn't need truly realtime collaboration if you used file sync as your backend, and gave up on the 2D sheet model in favor of a table/records model that lends itself to conflict free operations instead of edits to specific numbered cells.
Depends on your definition of dirty. I'd take "air traffic control running on emacs" over "guy prints out a form he received digitally and other guy types it into their computer by hand" any day. The first seems dirty, the second one is downright idiotic.
Yeah, I used to work at a large UK university and I saw (helped manage) a vast Excel spreadsheet which calculated students' final degree classifications. It was so complex I never managed to unpick the algorithms, just assumed it was right.
This being said, Excel is most likely the most widespread tool for anything that involves office work. I've seen entire financial algorithms being implemented using Excel alone.