When I was a young, opinionated software professional, I would have agreed with this wholeheartedly. But now that I'm nearly a couple of decades into my career, I've seen 1000 different workflows justified by "we're different, we're unique".
And if you ask how they're different so that they'd need a unique process to manage bugs, to manage invoices, to manage inventory, you get pretty much the same answers across the board. You hear about personalities and power structures and anecdotal preferences.
And sure enough, that's exactly what you see at the link:
Each person’s mind works a little differently, and each person remembers and processes information a little differently.
Yes, you're unique. Just like everybody else.
The fact is, the great tools that have achieved widespread adoption understand the problems, and all their caveats, better than you. Picking any established tool to manage information and workflow and getting good at using it will usually yield better results.
The search for the perfect tool or the perfect workflow or the perfect process always ends up being the enemy of good.
That's funny. I was the opposite. When I was younger, I would just say "lets default to other people's solution to similar problems and move on" but as time went by, I've become more "let's define our problems well and find surgical solutions by others". It's a slight difference but makes a huge difference in productivity.
Another way to say it is instead of taking someone else's blueprint, we build our own blueprint and use pre-fabricated parts to realize the blueprint.
In the end, sometimes solutions look very similar but there is that one little bit that gives you a competitive advantage that no one else has.
Years ago I analyzed the build vs. buy question and came up with a rule that I later found to be not so original: buy for parity, build for competitive advantage.
Most of any business is commodity stuff, typically things like HR, payroll, logistics, finance, security. Except if your business is one of those functions. Take Amazon's logistics for example. They aren't just any online retailer, they are the dominant online retailer in large part because they optimized the hell out of their logistics pipeline.
But if it's not your bread-and-butter, then yeah, go ahead and change your workflow to whatever COTS software you pick requires. You'll be fine. I worked for an insurance company, and they picked PeopleWare for their HR system and used commodity software tools for their actuarials. I worked for a very large shoe company they has SAP at the core of their (extremely complex) product lifecycle management system because their real business differentiator is their design and marketing.
That same insurance company had a customer sales pipeline tailored to a specific market – high-end professionals like doctors, lawyers, and similar – so they had custom software for that. That same shoe company had customized design and bill of materials systems so they could get their shoes to market quickly and globally.
I've been thinking a lot about the buy (or rent) vs build problem and one factor that keeps screaming at me is: who do I want to maintain it?
If my team or I want to maintain it, then build. If not, then buy/rent. I've built way too many things over the last few years only to realize again and again that things we build will need to be maintained by someone.
As you say, if it's a competitive advantage, then it makes sense for me to fix it, update it, adapt it, etc., otherwise it may be too costly to get distracted with maintaining it internally and easier to pay someone else to do that.
I agree with your rule, but I would add something extra:
Only build if you have the talent to do so. Early on Amazon managed to attract serious tech talent to the company. That enabled them to build. If a company can't do that then it's better to buy no matter what.
I would certainly hope that if the function is part of what differentiates the company competitively that they'd hire the talent. If they're trying to be the best "X" company and nobody who works there is any good at "X", they should fail. The dot-com era is littered with the graves of companies who thought that just having an idea was enough.
And yet it's surprising how a lot of companies simply won't match offers or try to play games with CoL. Or worst, consider engineering a cost center when it's central to the business.
> nobody who works there is any good at "X", they should fail.
Except almost every software startup in the last three decades was started by folks that were not experts in the domain they eventually became successful in.
I was going to say "yes!" but then I looked it up that Amazon actually launched before Ebay as a business. I don't remember actually using Amazon personally until the 2000s but they've been around since 1994, almost a year before Ebay. I'm struggling to think of any earlier online retailers who would have been building ecommerce software.
Especially in the early days, the main competitors to online retail were catalogue services. I would think they have most business and logistics problems in common, minus running a web site.
Joel was one of the voices that helped me understand that, while my analysis was useful for myself and the employer at the time, my conclusion was not an especially original insight.
In practice this isn’t a problem. Take a non-software example:
Driving instructors typically have their cars on a 3 year hire purchase then return instead of buying. Why? Way back when, my driving instructor actually bought one of the cars. 3 months later, the engine failed and it was out of warranty.
Because of that, he realised that paying a monthly fee meant he had no surprise failures, he returned the car just as the clutch was burning out (learner drivers don’t have great driving technique, weirdly enough), and had a fresh car every 3 years for new learners.
I don’t know the details but I’m assuming he’s using commercial lease hiring, or the U.K. rules are different. The mileage and wear and tear alone would mean he’d be unable to hide it.
Yeah I've never heard anyone think a driving instructor car was a great buy, unless you like replacing basically every driving component which can fail. These cars have it rough. Especially manual. I'm surprised a driving instructor (of all people) would think of doing that without also wanting to do a full engine and transmission rebuild.
It’s not a rental, it’s hire purchase with a warranty, so the most common things that are going to fail are covered. Other things are just not that likely to fail in the first 3 years.
There used to be a pretty big business in US of reselling expensive software licenses (think AutoCad etc). I remember Autodesk trying to stop it and not being able to which brings us to the current day.
Of course now that everything is under pay as you go SaaS model, that's out of window.
Agree. Interns or grads always want to throw whatever the most hyped tool in some domain currently is at every remotely fitting problem. Ansible all the things!
That's not to say you should always roll your own. This is one of those things that's really hard to become good at: Decide early on whether to go with an existing solution or create something from scratch. (and if you go for an existing solution, which one?)
There's many arguments for both sides, one thing often mentioned is that settling for some popular solution or framework will make it easier for others to get into your project, but if you bend over backwards to get three different tools to do what you want instead of writing a hundred lines of bash, you might be doing something wrong.
What I do is decide that I am in charge of defining the machine, and the tools implement the machine. The tools are a means to an end. If the tools don't exist to implement the machine, I write them. If they do exist, great.
> That's funny. I was the opposite. When I was younger, I would just say "lets default to other people's solution to similar problems and move on" but as time went by, I've become more "let's define our problems well and find surgical solutions by others". It's a slight difference but makes a huge difference in productivity.
I thought the same way except I was even more insane. I thought lets just teach everyone English and we won't have to do any internationalization/localization.
> In the end, sometimes solutions look very similar but there is that one little bit that gives you a competitive advantage that no one else has.
I'm all for doing something "different" if it gives you some kind of core competitive advantage.
But the pain of being off the beaten path can really exceed small benefits you get in unexpected ways.
Save real innovation and risk for the things that are core, unique to you, and possible sources of large competitive leverage. Be boring elsewhere, maybe with an occasional small sprinkle of something clever mixed in. Evangelize the small bits of cleverness, in hopes that someone else will start maintaining it. ;)
I'm not sure if I'm misreading you, you're misreading me, or we're on the same page :-)
Ideally, I map out workflow and find tools that fit the workflow in the Unix philosophy. Well-defined, do one thing well. As opposed to monoliths that purport to do everything.
My experience has been that doing things in this way builds a system that is extremely robust to significant changes.
> The fact is, the great tools that have achieved widespread adoption understand the problems, and all their caveats, better than you. Picking any established tool to manage information and workflow and getting good at using it will usually yield better results.
The stagnated companies with questionable data analysis because they're looking at beautiful SAP or other ERP graphs that have been filled with incomplete data by their end users on awful interfaces would disagree...if they knew.
I ask clients (and myself quite frankly): what is the competitive advantage of doing it this way? Do you get a leg up on your competition? What's the opportunity cost of spending time and money on this instead of your core product or mission?
"the great tools that have achieved widespread adoption understand the problems, and all their caveats, better than you"
Tools that get widespread adoption are mostly those that move quickly to add every little feature every manager/exec asks for in order to secure the contract.
Once they become incumbent, they become the standard, because managers are responsible for the 'big purchases'.
There's tons of very accepted systems that are 'not very good'.
It's more true now than it used to be. A lot of business software in the early days was an attempt to directly replicate paper business process on a screen. Enterprise software was sold as being low friction and not requiring users to learn anything new. And it was built by developers who started with a data model and then build screens to match.
I think we've finally tipped to the point that business processes are designed as digital first. The current gen is also following much more user-centered design.
You're absolutely correct: business processes are changing rapidly in a way that they haven't since the assembly line and Taylorism. We've had decades now of users and companies "paving the cartpath" by replicating paper on screen and learning what translates, what doesn't translate, and how to evolve. The workflows and the tools are evolving together now through a feeback loop, and that's a good thing. Consider order fulfillment and shipping for small to medium businesses today compared to the era before FedEx and eCommerce.
The military was ordering planes designed for the average pilot. But because there are so many dimensions along which someone can vary, most people are far from the average on some dimension, and when SHF and they need to be able to reach the right thing at just the right time, sometimes they couldn't.
Each pilot is unique, just like every other pilot. Accommodating that proved important.
Now... how well does that translate to building software? I can certainly see reasons why it wouldn't. But I don't see an argument that wouldn't apply to the planes as well, without actually measuring things.
> Now... how well does that translate to building software?
Well, let's dig on on what the Air Force ended up doing, overall.
First, they studied and improved the ergonomics. Specifically, making things like seats adjustable. The tool became more adaptable for a wider range of pilots.
At the same time, there is a limit to how far that can go, especially when we are looking at 1940s technology.
The Air Force thus imposed body-size standards for pilots -- 5'4" to 6'5" is a wide range, but it excludes outliers. Too short, too tall, too wide, or too heavy, and you couldn't be a pilot.
My Dad was in that category -- too big to fly his dream plane.
Controls were adapted so that unique functions had unique hand-feels. Flaps had a different shape of control than landing gear, and all of that was standardized across aircraft.
So what does that tell us about workflows for engineers?
Well, we can say firmly that physical ergonomics matter. Bigly.
Uncomfortable seats, poor-quality monitors, high-latency or slow network connections... all of those are deal-breakers.
But in terms of things like process and practices... standardize what you can, give everybody a sane baseline from which to work, and then empower them to customize as much as possible for their team or group.
At the end of the day, software is shipped by teams.
There's this odd, pervasive myth of the Morlock and Eloi in the software world.
E.g., that engineers are either beautiful artisans in designer specs to which every whim must be catered, or that you just need to shove a pizza under the door every couple of days and production code will then appear on your servers.
Don't open the door, though.
None of that is reality. And to work in a team, you get to surrender some of your personal work-autonomy, in order to leverage the cognitive multiplier effect.
> Well, we can say firmly that physical ergonomics matter. Bigly.
Can we? I don't think that's the right lesson to take from the story. The story clearly establishes that physical ergonomics matter when split-second reactions matter under unusual G forces. I would be astounded if we didn't find that they matter less in the sort of situations most of us find ourselves in while coding. It could still be more than "bigly"; it could be very little. We can pull in other evidence and experience to try and make a determination, but that's no longer drawing lessons from the story.
I think application to software development tools involves asking whether we find, there, too a high dimensional space; and then recognize that, in such a space (where each is picked from independent distributions), we'll find that each of us is similar in a lot of ways, and pretty different in a few. It doesn't seem unreasonable that this would sometimes impact choice of tool.
As you say, teams are a crucial part of the production of software; it's vital that whatever tools are chosen, they mesh well where there is interaction between individuals.
I also wonder if there's some twisted emotional flow in people to consider whatever hell they conquered as the way to go forever. Questioning their ways is seen as such an insult there must be something deeply wound in their brain.
I think you need to distinguish here between "core business" where you generate your competitive advantage, and everything else. In your core business, were you are going to make the critical difference that determines why your business or organisation exists, shooting for the most finely tuned, unique solution that maximizes that advantage - or at least, enables capability to do that - is critical.
For everything else - I agree. The more bespoke you decide to be the more pain you create for very little value.
Good explanation! These companies start out thinking they will make a better product than anything out there. But due to limited resources always end up with an ugly monstrosity that is hell to work with. So they end up with something more expensive (but that cost is hidden), and something worse than what's off the shelve.
I always ask them "So are we the only company in the world that is developing software?".
Tools are pretty powerful, so you build your process around them.
Spot on. Some things just need to be good enough. Spend effort on solving the problems with the biggest impact, your workflow and tools don't deserve that much attention unless it's completely not fit for purpose. Also changing your process and tools too often and too drastically usually makes things worse. People spend all their time keeping up with changes to how things are done as opposed to actually doing things.
I agree with you, in the context of working in a corporation. It's not a bad thing to pick tools and structure your workflows around them, because sometimes such tools serve as 'mental anchors' for standardizing a process.
The article seems to focus on personal productivity tools, for which the author's conclusion might relatively hold to a fair extent.
Yes. Just see any ERP implementation. Its better to take as many defaults as possible and go live, than spend years trying to implement your own convoluted unique workflow.
Process is a crutch for bad management. If you are constantly moving from process to process because they aren’t working, you likely need to replace a significant portion of the management team.
Almost all processes work well enough. And there is nothing wrong with trying to find an optimum one. But process is not the solution. Quality management talent is.
I cultivated an extreme dislike for managers for years. Then I became one. Things are a whole lot different from the other side.
You perhaps overlook a few things: (1) Managers are people too. They make mistakes. They should be able to recognize and correct them as others. Ideally quickly, transparently and with the insights of others. (2) Needs evolve. The right process for proof of concept R&D is not the right process for global domination with a mature product. (3) Management is about competing concerns. Often a decision must be made within a limited sphere of experience, time and resources and reviewed in the future. There is no 'right' decision, and it can be fiendishly difficult to quantify decision quality in many such circumstances without the benefit of hindsight. (4) Different people can respond very differently to the same management, so 'bad management' is perhaps too uni-dimensional a notion to hold much meaning.
I never said managers where bad or that I disliked them. Sorry you got that impression.
In this case all I was referencing was that frequent process changes and repeated failures of process are typically an indication that a change needs to happen in management not process. No where did I state management be eliminated altogether. I stated that better managers need to be brought in to replace those that are underperforming.
How unique are people? You take a bunch of teenagers, ask them to non-conform, and they all come up with the same pierced, hair-dyed fashion. As if they're clocks ticking at the same rate. If you have several million clocks reading the same time down to the microsecond, that implies the existence of a master clock they were synchronized to.
I implement workflows for a living and it is a million times easier to adapt your process to the tool than the other way around. You will spend more effort adapting the software than the people.
That's for business but I think it applies to individuals as well. You're much more adaptable than software. Pick the thing closest to what you want and change how you work to fit the software.
I work in professional services for a large IT vendor (and before that I worked for a place that used that vendor’s products) and I can tell you we spend a huge amount of time working with our customers to help them fit their workflow to our tools. Because there is no way we can engineer our tools to fit or enable every workflow.
Some companies have the skill and desire to build their own tools over our products so they can use their pre-existing workflows, but inevitably the abstractions start to leak and they almost always end up hiring us to train them on how to use the tool the way it was meant to be used.
I'm living that now a little bit. New tools and processes are discussed and evaluated based on how easily they can be customized, perverted, and eventually bastardized.
More frequently than you could imagine, people and organizations jump to how they are going to implement something before they define what they want to implement. And when the effort fails, the technology is blamed and they start all over again.
Sure but how much you gotta change is the difference between a successful implementation and a failure. Good tools are already close to what people do.
I’m not sure I agree. I think good tools represent the problem space well, and provide powerful ways of manipulating/interacting with it. That may or may not be what people are already doing.
I’d say solving a problem well is orthogonal to what people are actually doing.
I had heard this about Salesforce years ago, that companies generally do better by adapting to Salesforce than by adapting Salesforce.
In the end every quirky thing you do means more on the job training for all new employees. If you take quirky too far, then you can’t hire new senior talent (everyone starts as a noob). And it may be very uncomfortable to look at, but you need to think hard about who benefits by having things be this way. It’s not the company, and probably not the team. It’s a couple of people and likely also their manager.
Well put. And it's painful when either of these decisions goes wrong.
- When you write a tool without knowing exactly what is a beneficial workflow, and this imperfect, probably under-documented and under-tested tool must be maintained over the long term.
- When you adopt a tool - trusting that its developers know what a beneficial workflow is - and it doesn't suit your workflow, or skews your workflow unnecessarily, where you have to bend over backwards to suit the tool. It can add inefficiencies and cost of time/effort, that might not be worth it - but, once adopted, you could be stuck with it over the long term.
---
The posted article argues for the advantages of "homebrewed tools". The main point being, the priority and emphasis on the workflow itself, which the tools should support.
> I can build the tools that exactly conform to my workflows, rather than constructing my workflows around the tools available to me.
> ..Tools you build yourself can grow and change as your workflow changes over time.
> This is typical of the way I discover my workflows. I start with a minimal, bare-bones solution, and try to pick up on patterns and tricks I create for myself. And then I encode those patterns and tricks into the tools over time.
In my experience, I've found this approach to result in simpler systems that are easier to understand in a sitting. When it goes wrong, it can be adjusted right there in the tool.
And, in general, I feel that when things go wrong with an established, community standard tool, it's more painful to work around it since, usually, it's not easy or possible to change the tool to fit your ideal workflow.
---
There's something to be said for community-developed and supported tools, with detailed documentation and a crowd of people using it in production, essentially testing every nook and cranny of the tool.
I suppose only experience can truly distinguish when one approach is better than the other, in a given context.
In general, I agree with this... There are cases in which there is actual value in breaking "out of the box", but it's difficult to calculate the ROI. It also depends on the licensing cost of a tool and the skill of your team. Enterprise software can be insanely expensive, and with the modern framework and the right team, it might actually be significantly cheaper to build features yourself. I recently built a DocuSign replacement which also collects payment method for 1/2 the cost of the yearly subscription. Turns out they didn't need to actually collect signatures in the first place, but they were doing so because they modified their process to fit the tool...
ya really the only real cost was my time. It runs on their existing CRM platform so no additional infrastructure overhead. It's just a small react app which displays a PDF contract, gathers TOS acceptance and collects CC via a Braintree drop-in. They were sending contracts via Docusign (10's of thousands @ $11 per envelope!), then would still have to separately gather payment information and update the CRM to process the sale.
I spoke with there legal team and turns out they didn't really need a legally binding signature. New system does everything they actually needed, without any real reoccurring costs.
I came here to say this. And just mention that in most IT workflow disasters. It's usually when the company decides it doesn't want to bend a knee and change it's processes to fit the tool.
Absolutely! The majority of tools and systems in distribution are templates for effective workflows. For setting up your personal life as in this post, by all means go bespoke in your area of expertise or curiosity. It's certainly not the most effective or efficient workflow in the world, but that's not why you're using it, right? When performance really counts, always look long and hard at what the thousands-millions who've trod this way before have done before bushwhacking your own path.
While hiking around Iceland, for example, one should adopt established tools and methods, and adapt to them. No one's workflow can ever grow to accommodate that level of embedded efficiency and safety.
Real change in business requires adopting new processes. I just helped a small business shift from a paper driven workflow to electronic. Adapting their workflow to fit established tools was less work than doing nothing, less effort than their existing process; they doubled their output literally overnight. We'd have been fools to try to adapt a tool they'd never used or heard of before to look like their paperflow.
This sounds right but in my experience most people are set in their ways and it is very hard to rewrite the muscle memory of a process. Especially since it appears that most people are sensors (vs intuitive if you buy Myers Briggs take on personalities https://personalityhacker.com/develop-intuition-intuitive-aw...) then it requires a whole retraining of x does y and a does b. In my experience it is tremendously easier to have a computer change then a person much less a group. Admittedly most people when put in the position don’t have a choice and the cost of retraining is rarely measured but studies on adult learners generally point out it’s hard and doesn’t go as well as we’d expect.
> it is a million times easier to adapt your process to the tool than the other way around
In the short term, perhaps, but at what cost in the long-run? We end up with channels of accepted behavior dictated, undemocratically, by software teams.
The best experiences I've ever had is when you do a bit of both. I've mostly worked in areas where the steps in the workflow were designed to support some mandatory process steps for various compliance or legal reasons. Within a process step there's usually some give and take in the local workflow but most of the eventual users of the software are all too busy to spend lots of time learning new tools and tool workflows. So the solution?
1. Map these intraprocess workflows by talking to the users, find the biggest pain points for automation.
2. Build highly modular tools that implement those workflows and automate those pain points.
3. Deliver the tools into the environment and wait a bit.
4. Re-interview the users and find places in the workflow that are now the newest biggest pain points. Goto step 1.
It's a pretty simple process and not hard to do. If you do it right, over a very short period of time you deliver working tools that improve the user's day-to-day jobs and then just...continue to make it better.
Fairly often, after a few iterations, you end up just completely automating entire workflows and I've never had users get upset about this. It usually turns out those were boring, dull, and repetitive things they had to do and they hated spending hours of their days doing them anyways. Freeing them up from doing those things means they could move on to higher-value work that was far more interesting for them.
Note that the author specifically means personal (not shared) workflows:
> The Eureka moment that some of us feel when we finally find a notes app or todo system that fits our brains – that epiphany happens when the tools we use mirror the way our minds work, and how we want to move information through our lives. Good tools fit perfectly around our workflows, bad tools don’t.
> When we resort to having other people build tools for us, the tools they build might never quite perfectly fit our workflows, because they’re not built for our individual minds.
He goes on to advocate building yourself exactly what you need.
> He goes on to advocate building yourself exactly what you need.
This is what I've started doing. I needed a site generator, was too lazy to learn an existing one, and built something that would just do what I wanted, and no more.
It's actually quite a nice experience knowing a tool inside and out, literally.
I dunno. < 60 lines of a language I enjoy vs learning how to configure/install/write for a program I don't know. Honestly the former sounds like less of a pain.
I gave up on static site generators and use WordPress with default theme and a code highlighting plugin. I did start writing my own SSG but gave up on that too.
For simple things, I think I agree with what he's advocating. I made my own todo/tasks manager application that got my own needs. Took me 2 days of developing, and a few edits after using it for a while when I discover a bug or want to edit something. But I think for more advanced stuff ( a proper full featured bug tracker for example), I don't think it's worth the effort.
Yep - I started building personal tools when I realized that purchased tools are to benefit someone else - not me. Their goal is to make money for them not to make me happy.
Once I realized that I automate my own processes and I am so much happier. I went so far as to purchase an annual $99 developer license even though I have no intention of selling apps.
These kind of things are nice and all, and on a certain level I agree: shape your environment to do the hard work for you.
However, when considering overall productivity, it’s very easy to fall into the trap of yak shaving, and you’ll end up with doing a lot of work in order to be “more efficient”. This always rubs me the wrong way.
And it is for this reason I typically prefer some integration into a tool that I’m already using, rather than using a vast amount of tools that I need to integrate myself.
In the end I settled on emacs for most of my things here, and since I’ve been using it for over 20 years, the investment into learning ELisp etc pays of a lot. I much rather prefer to work with the framework that eg org-mode provides, than try to find some tool that is “just a little bit better!”
Yes, it may be better, but in the end it will involve a lot of work to tie everything together. I simply don’t have the time to do that.
I felt that that the author was more concerned about ownership than short-term productivity or efficiency.
With ownership a different kind of productivity is evoked: knowing how (and why) every part goes together the way it does. This could lead to a flow state when managed effectively.
I do agree that these exercises could lead into the realm of yak shaving, but in this context we're talking about making personal tools vs tools for others to use. I think there's a big distinction between these two approaches.
OP here -- I think you captured a lot of what I wanted to express. A lot of my DIY-ing is definitely about ownership: owning (1) my data completely, including how it's stored and backed up and (2) the future of these tools, so a SaaS provider can't suddenly decide next year that I'm no longer in their target market / most profitable customer segment.
Agree on the distinction between building for myself vs. others. There are a hundred things I'd have done differently, were I building for someone else.
Every second spent on developing one of these tools will more than pay for itself in your professional career if your professional career has anything to do with software development.
The time energy and effort will make the creator more efficient and more effective in nearly every project they undertake going forward in life.
Making and learning, especially if they are in areas related to what you do frequently (for example, as a career) almost always pays off in overall efficiency and productivity.
This sounds a bit like "the computer is meant to serve the user; the user is not meant to serve the computer." It's a valid sentiment, and one I use on my peers often, especially when dealing with end-user-facing software and systems.
But when it comes to Software Engineering Workflows, I've learned in my decades of experience that the progenitors of a process or workflow had no idea how to do it, made the assumption that no one had done it before (or that buying a solution would be too expensive), and then just cobbled something together to get the product built and out the door. They start hiring more employees who spend an inordinate amount of time just understanding the existing workflow and soon lapse into "well, it's not my problem - it produces output, and I just need to fix bugs / add features to the product."
Before you know it, you're ten years in, with a couple dozen white-labeling clients, and that build system can't scale.
What I feel like we need is an encyclopedia of workflows. So you wanna make an app: make several decisions now, and here's the recommended design workflow, engineering workflow, testing/staging/release workflows ...
Classically, that's a mistake. You end up automating an inefficient process. That's from manufacturing technology. The most cost-effective way to do something automatically at high volume may look nothing like the manual process.
Here's a good example. This is a machine making N95 masks.[1] It works about the same way someone would do it by hand. It's the obvious approach. It's painfully slow to watch.
Here's a machine making paper cups.[2] The process looks very strange. It's over 5x faster than the mask machine, and it's doing a more complicated job.
But this guy is talking about phone apps and stuff for people who work at desks, not automating a manufacturing line.
One big problem in the US is that there are not enough smart people making machines like [2]. And too many people making things like what the OP is talking about.
I love the principle of owning your env, using the right tool for the job (which may involve acts of creation)... but this
"Some of my apps, like Ligature and Noct, are written in Ink, which is a language I wrote myself"
is both impressive (truly next-level), and also completely out of reach / impractical for the vast majority. But it is inspirational; thanks for sharing -- your other blog posts look worthwhile too...
I actually believe fairly strongly that it's not out of most people's reach! I've written before [0] about my process for learning how to build a (simple) language, and the resources that helped me do it / how I approached it. Certainly one of my favorite projects, both intellectually and from a returns perspective.
There are so many great resources for building a toy language by yourself (my impl of Ink is a few thousand lines of pretty simple Go code) and it's never been easier. I really think most people overestimate how intellectually intractable it is. Building a production language? Yes, a lot of work. But a toy language for you to use / learn with? surprisingly doable.
The dichotomy between tasks and notes doesn't make any sense to me either. The distinction between short and long term things makes more sense, mentally and also technically - apps that have complex features to organize notes always load so slow that they're useless for quickly jotting down thoughts. But still I hate having my thoughts siloed into different apps. I hate having to decide what app to use to write down a specific thought, I hate having to decide which app to search. I made my own app[1], which is currently only useful for short term notes and tasks. I hope with Svelte I can add some features to make it possible to relate notes, to make it more useful for long term projects, while still keeping the app fast enough to be useful for quickly jotting down thoughts.
I love this. Very creative way of handling the search and persistence with the fulltext search to have it easily work as a todo (or any other "tag") app.
I wouldn't use it in a browser with local storage only but it actually makes me want to copy these ergonomics for my personal use with some sort of syncing..
Ctrl+Space to toggle search results is not suitable for a lot of people. That's the default keybinding for spotlight/launcher style apps. I personally use Synapse, so unable to toggle search.
I ran into this problem when editing videos. I automated some repetitive tasks that make life easier.
I have a scripts to move clips off of SD cards into a date sorted folder structure. I have three cameras and a phone, and usually want to get clips from the same date on different devices (SD cards) into one folder on my backup drive.
There's also a problem when you film yourself, that lots of footage is wasted without action. Yesterday I recorded a video that was 10 minutes long, but only used the middle 2 minutes. So I used a script to chop up the video, then deleted the useless parts.
I also use Typora (mark down editor) for planning and notes. Highly recommend it.
The headline doesn’t really reflect the post body. The author is advocating for writing your own software tools for your own personal note taking and organizational use.
First off, their tools are very beautiful from the screenshots, and if they work for them great! However I can only think of the time liability they are (they even wrote the language they are written in). I can only imagine how much time this person sinks into making them. If that’s what you love doing great! But for the vast majority of people that’s not reasonable. They are pretending that it’s a high productivity approach. It’s not.
Productivity = results / time
Spending a few dollars (equivalent to working for an hour lets say) and getting a high quality tool that would take you months to build yourself, is the high productivity solution. Many tools allow a high degree of customization and plugins, which allows people to really make it their own as well.
OP here -- that's definitely a valid argument. If the only reason to do this were raw productivity gains in my work, it /might/ be overkill. I also find it personally satisfying to build and run / talk about DIY tools (and I'm young, I have lots of free time), and there've been lots of extra side-benefits to making a habit of it.
For what it's worth -- it doesn't take me much time to build these tools. Most are an afternoon project / a weekend project. So that also makes this more reasonable for me.
EDIT: I'm also no stranger to off the shelf tools -- I've shopped around my fair share of todo lists and notes apps :) Just haven't found any that worked the way I liked exactly and allowed me to own my data the way I wanted to.
There are so many todo list apps, maybe the problem isn't building 1 true todo list app but rather a way for everyone to build their own todo list app.
When once upon the time, I decided I was going to write a novel, I spent ages finding exactly the right tool that would do the typesetting exactly right and had all the features I just KNEW I needed. Never ended up writing a damn thing.
Eventually, popped open Notepad and actually did some writing. Felt amazing to just get started.
My rule of thumb for software tools: first do something the annoying, hard, manual way. Then introduce equivalent third-party tooling once you understand how those tools work, why they help, and what problems (in the manual workflow) they are trying to solve.
It's super cool to see this kind of self-sufficient toolmaker mentality can still thrive in today's world of big, interconnected, and multi-disciplinary software teams. It feels like a throwback in a way, but also it highlights the power of simplicity and YAGNI. Of course the fact that the tools are personal makes the tradeoffs easier to reconcile, but even as you start to scale and generalize it's worth questioning if a simplified, less feature-rich solution can generate more leverage to your particular problem.
Nicely written. I think the "Importance of Ownership" is the prime reason for doing this, but most people wont see the benefit (until it is too late when they do lose something)
Yeah and that’s even more important in enterprise where software vendors could lock you in and put your features requests or bugs in some “ideas” queue and leave it there for years. Or when software vendors cripple your data strategy by closing access to all data users produce in their platforms. It’s a shit show when you depend on not so great vendors for critical business processes.
If you're working with a framework or tool, you should be as idiomatic as possible. Follow the best practices of that tool. It's not because you can't do it better, you might be able to. The reason is because when someone inherits the project from you or you need to onboard someone, all of their prior knowledge will apply to your build, and all the documentation and help forums for that tool/platform will actually apply to your project.
Part of this is ego I think. Many of us have landed jobs in agencies or at companies who produce commodity software using a framework or tool, and we see things about them that we feel we can do better. But if you're working with a client and they've decided on a framework, part of that decision is the implicit understanding that you will build them a platform that every developer using that framework can work on.
It's not unlike choosing to use Volkswagen vans for a transport business because you have a good community of Volkswagen mechanics. If they ask you for a VW, and you give them a VW that has a Ford engine and a Nissan transmission, they're going to be pretty confused and annoyed when the mechanics tell them they can't work on this and need to rebuild it.
This is excellent stuff. It fits right into the hacker ethos of hacking to serve your purposes instead of settling for tools that probably won’t just because they’re more convenient.
If you don’t mind me asking, how much effort did it take you to develop each tool, and did you find that as you did more tools, subsequent ones took less time to develop and/or were more capable?
My answer your second question actually feeds into the first --
These days developing each tool is 50% taking pieces of tools / libraries I've built before and integrating them together. I've implemented a data store, a form UI, a websocket server, etc. each a few times so as I built up a collection of things I can borrow from other past projects, new projects take less time (or, I have more bandwidth to tackle more complicated projects, new a new tool if need be, etc).
I have another blog that goes into my side-project workflows[0] but in gist, I try to make a "usable MVP" for each project within a single chunk of time -- usually a weekend. And from there I try to dogfood it and add changes that I need to keep using it.
All this is helped by the fact that, at least with these tools I use, I try to keep a really simple stack. Go or Node.js server running on a single VPS, light frontend using my own framework [1]
This is an incredible amount of work for personal tools!
Do you ever miss features from traditional note apps, CRM, and web page builders? Does tool maintenance get in the way of your work?
I do agree with your argument about building tools to fit your workflows. Low code and no code tools will bring down the cost of custom tools dramatically.
Companies don't update their tools enough because people hate changing their workflows. I've worked with customer support teams and change in workflows is a big reason why they don't want to buy a 'better' SAAS tool. They'll lose productivity in short-term with a new product that's different.
>Do you ever miss features from traditional note apps, CRM, and web page builders?
Very rarely, and if I do I build them in. But my needs are pretty low -- I'm ok with plain markdown/text for storing most data, or nicely formatted tables. I've found that super-rich document formats with embedded this-and-that don't actually add a lot of value for me (but of course this might be Stockholm syndrome speaking, I suppose).
>Does tool maintenance get in the way of your work?
No, and this is a big consideration for anything I build. I try to design / deploy apps so they require minimum maintenance. Static Go binaries + going slim on frontend dependencies both help with this. The only real "maintenance" I have to do is periodically reboot my Linux box and upgrade the Ubuntu version to the next LTS every few years. All my binaries are deployed as auto-restarting systemd services on an small VPS -- not depending on more serverless-style platforms helps keep maintenance needs low too, I find (though there are disadvantages, it doesn't really apply for one dude running a dozen web apps on a single server).
There’s a balance to be struck. If you’re doing something that tens of millions of other people are also doing and someone has even half-thoughtfully built a tool to address the common workflow of those 10s of millions, I’d think long and hard before concluding that my use-case requires/justifies a different workflow.
Nothing like building new software for a client and having them slowly change every element to work like the old software, which they hired you to replace because it was terrible
In my experience, most folks aren't sophisticated enough to build their own tools. Having said that a workflow tool that has a default workflow that most users can adopt but also some flexibility that allows sophisticated users to adjust it to their needs without making it super complex to use is a win-win situation. I founded a company that solved a workflow problem for K-12 schools. Our initial launch was pushing school admin users to just adopt a single workflow. Most smaller schools were OK adopting it but as we went up market, we realized that even if they want to, it's very hard for bigger organizations to change their workflow (people, processes, bureaucracy). Slowly we started making the system a bit more flexible allowing them to tune it to their workflow needs.
I have always longed for the graphical version of the Unix philosophy: rather than running large monolithic "applications", you have a lot of single purpose GUI utilities with well defined input/output interfaces that can be chained together into workflows using a simple script.
For example: append some markdown text into a given text file; open a text file, tokenize and filter lines where token 1 matches user input; if the user input matches a certain pattern, route it to a given file etc.
I'm already doing this with bash scripts (and they work for me), but for this idea to take hold with non-technical people, we will need GUI utilities.
The thing that feels implicit, yet false, in all of this is that workflows are static. People evolve, learn, change. Teams even more so.
The most important process I implement with teams is retrospection and its use as a mechanism to modify process in a principled way.
So as you learn about your own process, your tools can either fall away and be replaced, or flex with you and stick around for longer. My guess is that flexibility is slightly better than fixity, as it'll let you live within one ecosystem for a little longer.
The other way of interaction is to be inspired by new tools to modify your process. This feels pretty off to me, at first, in that it means that you're moving your process do to exogenous things as opposed to endogenous learning.
But that sort of work can be really valuable too. It's good to learn other people's methods. It forms an impulse that can get you out of local optima. It's an important part of the learning process.
I have to say, I strongly disagree with this. As a professional software developer I've watched legions of clients fall into the trap of customizing tools to adapt to their specific sacred workflow. It always ends up being expensive (though admittedly profitable for me) and most importantly, it always ends up trapping the client in a situation where they cling to the customized tool for far longer than they should.
The moral of the story: Your workflow is not sacred. Even the way your brain "works" is not sacred. Learn to adapt or be prepared to die. If your workflow isn't changing or at least being tweaked from time to time, the odds favor the idea that you have grown stagnant. That's not to say that change for the sake of change is the preferred alternative however. I'm only saying that treating your workflow as sacred and hence immutable is a dangerous trap to fall into.
In my experience (I have a bit of that), tools tend to be whatever I need, at the moment, to get the job done. If it works well, I look to use that tool again; maybe finding a way to make it repeatable/scripted/generalized. Some tools have lasted a couple of years; seldom have any lasted for much longer. I can't afford to marry them. It's just a casual "friends with benefits" relationship with most tools and techniques.
Tech changes quickly. Not just the technology, but also the markets, customer bases/expectations, design ethos, and delivery/distribution systems.
So does the jargon, but I've learned not to waste too much time trying to keep up with that.
If I get too mired in the workflows, infrastructure and tools, tech can go whizzing past, while I'm polishing my scripts.
I agree with a lot of what's written here and applying it to CI/CD was what led me to build https://boxci.dev
In my view your tools should come to you - for instance with CI, why should you have to go all-in porting your production build pipeline for a third-party platform and workflow when you have a build script that already works, and fast, and you really just need to change a couple of flags to build for production rather than dev? As the author says, you shouldn't change your workflow for the tool, the tool should help you with your workflow.
If you don't have a workflow, build your workflow around the tools you think you need. Then start the "build-refine-use tooling to aid the workflow that you came up with based on the tools that you're using that determine part of the workflow you have to best fit the tools that you use to help your work flow" dance.
If you already have a workflow, you're already more than familiar with that dance, and there is no advice.
Sometimes your work flow is shit and you need to change it. Imagine a carpenter telling you that his thumb hurts a lot when he uses the hammer to punch in a nail. Because he likes to keep his thumb on top of the nail. That's his work flow.
Would you spend effort designing a novel kind of hammer that grips the sides of the nail instead of striking the head? No. Just tell the carpenter to stop doing the stupid thing he has been doing.
What's trippy is whatever workflow you have is probably molded around tools and biases that you take for granted. In modern times, there is no workflow that exists independent of technology. I can't imagine a way to isolate a "workflow" from a toolchain. And at that rate, you might as well just sample different tools until you find harmony.
I want to agree with this author, but sadly the business world does not agree.
The reason Airtable, Notion, Monday, and others are so successful is because they do precisely the opposite of what the author is suggesting.
This is the same reason most workflow software starts out as a replacement to Excel and Spreadsheets, but people continue to resort to Excel and Spreadsheets.
The problem with "Build tools around workflows" approach is that everyone has a slightly different workflow, so you end up with tools that must be customized around workflows, which results in extra and unnecessary spendings.
This is great for software developers because it guarantees work, but bad for business because of unnecessary IT costs.
Dear Mesdames and Sirs, is this the Topic about a mechanistic Worldview and the mechanistical-Thinking, i have heard about ?
And an other kind of... diversity ?
In front of this screen, polarized, asking myself "Do i became too choosey ?" For what Narcissus -the protagonist is meant, to "think of oneself" is any good for, not ? P-:
Not all tools are designed the same way. This advice sounds good to hear but good luck finding the tools that can be customized to your workflow.
I always found that it is easier to make small changes to the workflow based on the tools and the tools will increase throughput and will justify the effort for the change.
What have you done with all these tools, so fitted to your workflows? What, other than the tools, have you made? How have these tools helped you build a better world for other people? There are new tools, now, yes, custom-made to fit your hands, but what of the rest of the world?
Hey natcombs, OP here :) Just fixed up the link so it should point to github.com/thesephist/lovecroft which is the correct link! Thanks for pointing it out.
If we're just thinking about it in terms of pure money optimization, that makes sense. Why spend 170,000 dollars when you could spend...nothing.
But it also seems the author actually enjoys building these tools. So it doesn't actually cost him anything. The work itself is a reward instead of a cost.
I can't say spending 1700 hours making my own tools is what I really want for myself, but I guess different people are different (I'd hate to find out exactly how many hours I've spent editing ~/.emacs though, tbh...)
And if you ask how they're different so that they'd need a unique process to manage bugs, to manage invoices, to manage inventory, you get pretty much the same answers across the board. You hear about personalities and power structures and anecdotal preferences.
And sure enough, that's exactly what you see at the link: Each person’s mind works a little differently, and each person remembers and processes information a little differently.
Yes, you're unique. Just like everybody else.
The fact is, the great tools that have achieved widespread adoption understand the problems, and all their caveats, better than you. Picking any established tool to manage information and workflow and getting good at using it will usually yield better results.
The search for the perfect tool or the perfect workflow or the perfect process always ends up being the enemy of good.