Me: This very specific thing is broken in general for your product when I do a and b. Just like users x,y and z report on <random forum>.
(Siemens) Support: Before I can help you, please find the serial number via <tedious procedure>, and the exact version of subsystems <x, y and z>.
Me: Here you go, though I fail to see how my specific setup is relevant as the problem has been reported on forums for years, and it is easily reproducible
Support: Please update to latest version x.
Me: Version x has known regression which will break the machine for the customer. I did the 1 hour procedure anyway but the issue is still present.
Support: Please execute this <obtuse command that runs a trace> and download the log from <airgapped machine> with SCP
Me: O well, did that here is the file. Don't understand why you can't run it on your machine
Support: Please try <irrelevant thing>, reboot (wait 5 minutes) and run the trace again
This is me almost every time I interact with ISP support.
Except… recently I completely misdiagnosed something. So while I was getting politely frustrated with the support clerk, he was stepping me through a set of irrelevant seeming procedures which, indeed, resulted in identifying that a piece of hardware was broken.
In this case it was the fiber to Ethernet adapter my ISP uses. He needed me to verify that, at every step of the way, pieces of my infrastructure were not the cause of my flaky connection (they weren’t). However, as a final step he had me reboot the adapter and it didn’t start back up. Turns out this is a rare failure mode and the flaky network I’d been seeing was an early, year long, symptom of this issue.
I had DSL with a router and modem. I never had problems with it, never required reset etc. Over the years somehow I forgot and thought it was a combo router and modem.
Then when I finally had a problem, I insisted to customer support I had done many power cycles already. They scheduled a tech to come out, but I realized my mistake before it went that far at least.
I went as far as going out to one of the local metro offices for my ISP in the city, to demonstrate some bizarre problem that I was experiencing with my combo router / modem.
I was actually able to demonstrate the issue in person, and everyone was scratching their heads trying to figure out what was going on.
I decided to try factory resetting it as a hail mary... and instantaneously I got a perfectly functioning ADSL connection.
It would have been far less embarrassing for me to have thought to try that and make the effort at home. lol
Is Support just completely disconnected from Engineering? Do they not have a way to report issues and indicate that many customers are having a specific issue?
Does the company believe that giving a customer a runaround will make them less upset than saying "Sorry, this is a known issue. We're working on it but do not have a timeline"?
Certainly at some point, some support person is going to be like "Huh, we have a lot of customers complaining about an issue, and our usual flowchart script doesn't seem to resolve it" and try to work it up the chain, right? Or does it get to their manager who says "Meh, that's an engineering problem, not a support problem. Get back to your tickets!" and never pass it up?
If you're working with a large company, Support is outsourced to a bunch of people reading scripts in Manila/Bangalore, and the external company employing them is actively incentivized to never resolve the root cause of any issue, because doing so would mean less tickets and less billable hours.
In my first job in a software maintenance project, I was overenthusiastic and was closing issues left and right. My manager called me to his office and told me that if I solved all the issues, the project will need much lesser people when they renew the contract, which might lead me, the junior most person, to be sent to the bench until they get a new contract.
Sounds like something that could be turned into an opportunity. Immediately start looking for a different job, and on the interview explain “I left previous job because I was too efficient at solving problems. After X time, there were no outstanding issues so I was no longer necessary. I’m looking for the next place where I can make a difference”. There are plenty of places that see fewer issues a positive.
To give you the other side of this, I have had service providers obviously do this to us. It didn’t result in more billable hours at renewal. We just systematically moved teams leadership to a competitor - we have found that mixing companies of origin in teams keeps everyone more honest - and removed developers we thought were not meeting the productivity standard we were looking for.
Cheating your customer is a dangerous game. They are not dumb.
It’s not that we were inefficient, we got some 80+ tools to support without any knowledge transfer or documentation. We got regular issues, and I was digging to find the root cause and solve them for which apparently we weren’t paid.
This is the usual response when a companies customer numbers start to scale up so far that the volume of users like the ones in Vegenoid's parent comment start to overwhelm the support staff. Keeping up the good/decent customer support that you could give to your first few hundred or perhaps even few thousand users eventually becomes ludicrously expensive or even impossible.
Then the original article's "Keeping this running and supported is shit. People are idiots and time wasters! Automate all the things!" stage kicks in.
So "first level support" is created, who's main objective is to get rid of support requests with the minimal effort by the least skilled staff possible. So everything is written into a script that call centre employees are required to follow. Low skill minimal wage staff are required to ask stupid things like "Have you tried reinstalling Windows?" and getting a confirmation that they have - before any support request is passed on to even a junior or intern developer. At this stage nobody gives a damn about users who need help, and they outsource that work to other users on the "community forums" and the entire support team is fired.
(Google, of course, being a world leader in both webtech and customer acquisition, completely skipped the "provide decent customer service" stage and went directly to the "don't give a damn about users" end game.)
Yes. Engineering time is expensive. Support exists to resolve problems without needing engineer time except when the company thinks that the problem is worthy of being addressed.
The tendency to wall off engineers is often taken to a counterproductive level.
Obviously, not every support ticket requires engineering attention. I would wager at least 90% don't. In the case of B2C businesses, likely 99.99% don't, considering 80% are customers that simply need a password reset and can't figure out how to do the most basic Reset Password -> Open E-mail -> Click link -> enter new password flow.
But there's a line SOMEWHERE. The few tickets that DO need engineering time REALLY NEED IT, and it's completely asinine that in some corporations, it's impossible.
If, for example, 100 people are talking to support for the exact same crash, and people in support are able to reproduce it, it needs to go to Engineering, rather than support telling customers "Have you tried formatting your drive and reinstalling everything from scratch?" when they already know it won't fix anything.
The best place I ever worked had fairly sophisticated, formal channels for noticing when a cluster of related-looking problems happened repeatedly, and escalating that to engineering for a fix. It also had open Slack channels with engineers and product managers in them, and some informal understandings about what type of problem was appropriate for support or professional services to bring up, there.
That combination kept the customer-frustrating bugs quite low, while still allowing engineering to keep developing new features at a fairly rapid pace.
(and then we got purchased by a PE firm and dismantled)
I think sometimes the company doesn't know what to do or doesn't care. But they have to make some response. So they just ask you to do a load of busywork, to keep you out of their hair for a while.
The other side of the coin is that support gets an ocean of lazy/unrelated/confused support requests that the engineer cannot help with, and responding and explaining that eats up a huge amount of time. I don't know that I agree with putting up barriers to reach support, but they are trying to address a real problem and I get where they are coming from
i remember we had similar issues at my 2nd to last job a bit ago. we would often times try to reduce the friction, but it built up and built up until, one day, there was zero chance of reaching anyone worthwhile. i wish we had gone back and really focused on providing as much support as we provide in product!!
sometimes we're aware of an issue and management has decided its not worth the effort to actually fix it, but also management doesn't want support to straight up tell people we wont help you, so they basically obfuscate
> Is Support just completely disconnected from Engineering?
Support is almost always tiered because $$$. In an ideal situation (hello GitLab!) tier 1 they are friendly and competent triage artists that can redirect lost customers and handle the common basic cases. Tier two is essentially an experienced and skilled tier 1. It's not until you get to tier 3 that you reach an engineer, usually one dedicated to support. That engineer is the one who reaches out to the operational engineering team if needed.
These flows are intended to dissuade problem reports, because the business is too big to care.
I once reported a ui bug to Discord(the game chat). This is how it went.
- me wrote an elaborate email and screenshots of the problem with step-by-step guide to reproduce it
- support responded me by asking build version, my os version, device type
- me screenshot all the requested details + manually wrote down in response email so someone can also copy-paste somewhere directly from my email
- support responds by asking me to clear my iphone cache(sending me a guide for Android's App data cleaning process) and see if issue persists
- me respond that it is not Android and since I knew what they will ask, I have uninstalled, reinstalled, logged-out, logged in and documented the whole process by recording my screen while doing it
- support, please try to logout, then uninstall and reinstall again
- me begrudingly do it again(record screen), send them everything
- support, could you try updating your device OS?
- me check for updates, iOS says it is latest, me send screenshots
- support, can you try disconnecting and reconnecting to your network or rebooting your device?
- me follow the steps(by recording my device using another device) and send it back
- support, can you try factory resetting your device
- me get pissed off, I mean c'mon, the issue is that, the client is incorrectly handling the on-screen keyboard events and has nothing to do with my device,
but giveup anyways and write on twitter that I tried to report an ui bug, is there some engineers I can reach out to?
- twitter official DM me and what do you know, it was the same person who was responding to my emails and tells me, if I tried resetting my device to factory default, else they'll close my ticket as not reproducible
I just gave up and uninstalled discord. I mean, sure there are lot of useless problem reports, but when your user goes on to extreme lengths to document an issue and cooperates, please take it seriously.
In most cases, the whole script is intended to deter the less patient consumers/users and not actually solve any problems. In some cases, the support is just an outsourced team with no connect/contact with actual team or the product.
As mentioned in another comment, support shouldn't be entirely disconnected.
If a customer seems to have discovered a legit bug in the software, there needs to be a path from support to engineering for the bug to get reported.
At the very least, support staff should be willing and able to attempt to recreate a bug that a user is reporting, rather than asking them to completely wipe their device and install everything from scratch for something that is easily reproduced.
Let's take a very simple example. Imagine you've got this Chess app, and whenever someone creates a checkmate involving two bishops, the game crashes, 100% of the time, but only in cases involving two bishops. Sure, maybe the first ticket you get, you go through the workflow of reinstalling, etc. But by the time you've got 100 tickets all saying their game crashed on a checkmate involving two bishops, that should have been escalated to engineering. At the very least, support should be saying it's a known issue. Honesty is going to be a lot less frustrating than to be told to take steps that both sides knows won't fix the issue.
What do you do if you are incorrect? It has happened many times that a customer calls certain they know what the issue is and even if the customer is well versed in that field he may be wrong.
I hop on video and do screen share. It probably takes less aggregate time, they think they get special treatment when really I'm just saving time and I know how my stuff is being used more.
Them: Don’t close the ticket, I hadn't had a chance to check that yet.
Life gets in between and this one library or project is usually hardly the only thing that person juggles. We need to accept, that sometimes issues remain open for an extended duration. The worst is, when you have the same error or issue someone else had already, but their issue got closed by an effin github bot, that automatically closes issues, because someone hasn't replied for a day or two. Like, you are not the center of the other person's life. Just like no one forces you to work at no cost for others and help them, they should not be forced to give undivided focus to your project's issues.
Having bots close issues, accompanied by a rude automated message is often contra-productive. It would be fine to instead post a reminder in the issue, asking for an update like shown in the example:
Me: Just checking in to see if this is resolved?
This is actually a very polite form of handling it.
This. Usually if you are looking at tickets closed, that means you are using that as a metric, which is a BAD idea. Ticket lifetimes and movement are more appropriate metrics.
What I find really infuriating is when I was the last person to post on my issue, and I'm waiting on an update from the maintainer, and the github bot threatens to close the issue on me for not posting any updates!
I've started replying to it with "No updates here." to reset the timer.
I almost never see bots close issues that are less than 30 days old. Many projects can change a lot in 30-90 days and the bug may no longer exist, keeping issues open when they may no longer be relevant isn't helping anyone either. If it is still relevant, it can simply be re-opened. I don't see any downside to semi-aggressively closing stale issues. If it's easily reproduced then most good projects will mark it so that it won't be auto-closed.
> If it is still relevant, it can simply be re-opened.
I've seen places where tickets were not allowed to be re-opened. If a ticket was closed for any reason at all besides a misclick, a new ticket had to be opened with a link to the old ticket if necessary.
Automatic closing tickets is a solution looking for a problem. Humans should close tickets when they're resolved or no resolution is anticipated or planned. Putting an arbitrary number of days on it with a one way trip to the [closed] tag is like following a broken clock because it's right twice a day.
Usually it can't be reopened because you can't even really get someone to look at it, because issues are wrongfully closed so frequently that they don't pay attention to complaints about the closures.
When I see some bug like this I do wonder why don't more people fix the issue themselves or think that it might be specific to their setup or accept a little random lag.
If I received a bug like that I would immediately think why are you telling me this... just fix it yourself and share your fix if you want. I probably have higher expectations from my users. You give the software away now they want you to fix it for them.
> If I received a bug like that I would immediately think why are you telling me this... just fix it yourself
Then you're part of the problem.
I am far less equipped to handle a bug like this than you'd think. It would take me so much more work and time than asking someone who already knows the project and how to work on it.
If I did this for every bug I reported, I wouldn't have a job because I wouldn't have any time left for one.
You know, this is also my issue with Linux. The attitude is generally that if you want to run Linux, you are expected to do anything you need completely on your own, including fixing bugs. This is why macOS is my preferred operating system. (Not that I can run it right now...)
I'm of course not entitled to anything from you (or anyone), but the one thing I won't do is fix it myself.
I don't really want to be a hacker right now. I've tried. I can be, but it rarely pays off for me. Not even financially, just emotionally.
I had this experience with an online game crashing. This is a game I play all the time, which recently recieved an update that adds a kernel level anticheat program. The game would start and then black screen, and I would get a null reference exception popup. Audio and input would continue without any graphics. To me this likely indicates a lack of init result check on some pointer that gets passed to a graphics init function, and then they try to access the pointer that was never assigned to. (my guess anyways)
I was punushed for "leaving" the game, banned for a couple weeks, and sent an obnoxious email about my bad behaviour. I reached out to them on customer support handfuls of times and was always given a list of three steps such as "upgrading graphics drivers", "ensuring i disable my firewall", "using a pc with high enough specs" (my machine is ridiculously overspecd) and things like that. I would follow the three obviously pointless steps, and then be given another set of obviously unrelated steps. This went on about five times over a month.
At some point I just told them, look im a fucking game developer, give me a log dump tool, or a build with debug symbols and I will fix it myself, and finally a real human showed up and actually sent me one. I recreated the bug and sent log captures but they wouldnt let me help beyond that. Then at the end they still scolded me for "leaving" the game.
I dont blame the average person for expecting customer support to be unhelpful or a robot by default, and not following steps.
We had a Salesforce consultant who was like this. I'd send him a list of five things he needed to do to make his jitterbit connection work with our internal API.
He'd do one thing, report that it still didn't work. I'd ask if he did the other four things. He'd do the next thing, report that it still didn't work, etc.
God only knows how much that department was paying him.
_This_ is exact the place where a LLM could step in and could really help:
1. Customer describes the problem
2. You get the problem, parsed by the model, including a summary and a suggestion what could help.
3. You write "do that 3 things"
4. The model tells the customer to do the 1st, then the 2nd and so on
5. The model reminds the customer in intervals that things are not completed
6. the model notifies you after the 3 things are done and the issue is still there.
You don't need an expensive, opaque, antisocial system to run interference against your users. What you need is literally just a checklist.
Give the user a checklist instead of just freeform text -- where they can check off the three individual things to try. When they've checked them all off, they can go back to the well for another response from support.
Me: Please try these 3 things and let me know how it goes: (list of 3 things with instructions)
Them: I tried (thing #1) and it didn’t work.
Me: Thank you, please try these 2 things and let me know how it goes
Them: I tried (thing #2) and it didn’t work.
Me: Thank you, please try this thing and let me know how it goes
Them: (no response)
Me: Just checking in to see if this is resolved?
Them: (no response)
Me: I’m closing this ticket as I haven’t heard back, let me know if this is still a problem and I can reopen it
Them: Don’t close the ticket, I’m still having this issue
Me: No problem, can you try this thing and let me know how it goes?
Them: (no response)