When calling That Friday feeling is back with us after we spent a week dealing with IT problems and avoiding the boss’s gimlet gaze. Hopefully you didn’t need to do improvised debugging in production. Welcome to on-call duty.
Today we look forward to the return of Who Me? Author Susan, who previously spoiled us with a story from two decades ago.
Susan’s latest anecdote takes place in the months leading up to Christmas 2001 when she was suddenly unemployed. She had enjoyed the lucrative life of a Visual Basic 6 and SQL Server contractor before her four-year-old employer suddenly went bankrupt and jeopardized her dreams of a lavish holiday in the Scottish Highlands.
As was customary in those happy days, she didn’t have to wait long for a new role to fall into her lap. A few days before she was due to leave for vacation, she had a phone interview and was parachuted on a rescue mission. Your new employer needed an online web application; her predecessor had been fired after repeatedly promising everything was going according to plan, and when the deadlines began to emerge it had become clear that everything was not going to be okay.
As for the interview, Susan recalls the crux of the matter: “The IT manager asked, ‘Our previous developer couldn’t successfully deploy the binaries – do you know how to do that?'”
Her (rather smug) answer was, “It’s pretty hard to screw it up.”
With the confidence that only years of contracting can bring, she walked into someone else’s office, surrounded by strangers, and started the project. Everything looked fine, except for the previous coder’s inexplicable inability to provide the Visual Basic binary. It took her five minutes to figure out what the problem was.
Two words to scare many VB programmers back then – “binary compatibility”. Enabling the option would ensure that the output binary file is compatible with its predecessors. The setting wasn’t checked, so she turned it on after checking the interfaces. The problem was solved, “so it was live on the same day.”
“If there ever was a case of ‘it works on my machine’,” she said, “it was.”
But what about the on-call moment? Susan’s original contract was for three months. So she was there when the web application finally went live. It was received with a lot of jubilation and celebration …
… until it stopped working.
The expensive contractor who saved the project was asked again to save it a second time.
She checked the logs. She checked the code. There was no reason for the web application to lock, but it did, even though it passed all tests in the test bed. With some embarrassment, she finally decided to go where programmers fear and check out the physical servers.
The problem was visible the moment she went to the desktop of the attacked web server.
“Someone had VB6 installed on the production web server,” she told us, “and it was just in the foreground, stopped at a breakpoint.”
In a desperate attempt to solve the deployment problems, the previous developer had decided to put a full version of Visual Basic 6 along with the source on the production web server.
“I’m sure the previous developer never informed his boss that he did such a monumentally stupid thing!” she said benevolently.
The immediate problem was addressed by removing the breakpoint and pressing F5. A few strong words were later spoken to the IT experts who were responsible for the server themselves.
And the bigger problem? Well, he had already been fired.
Have you ever been called to fix a problem that was heavily someone else’s work? Or were you the person who thoughtlessly planted the stink bombs because you knew you would be the one who was called out? Tell Your Story On Call. ®