New Jersey needs volunteers who know COBOL
April 9, 2020 4:35 PM Subscribe
Yay. A place for us old geeks to ramble. Mainframe didn't even used to be synonymous with IBM. I got my start on a Honeywell DPS-8. (Which were born as General Electric hardware before Honeywell bought them.)
posted by bcd at 4:46 PM on April 9, 2020 [4 favorites]
posted by bcd at 4:46 PM on April 9, 2020 [4 favorites]
So are a lot of Medicais systems and the Cook County courts.
People still have paid full time jobs maintaining COBOL systems.
posted by crush at 4:51 PM on April 9, 2020 [4 favorites]
People still have paid full time jobs maintaining COBOL systems.
posted by crush at 4:51 PM on April 9, 2020 [4 favorites]
I was debugging JCL today.
posted by tofu_crouton at 4:55 PM on April 9, 2020 [8 favorites]
posted by tofu_crouton at 4:55 PM on April 9, 2020 [8 favorites]
I hope you remembered the cover sheet on your TPS reports.
posted by RobotVoodooPower at 4:56 PM on April 9, 2020 [11 favorites]
posted by RobotVoodooPower at 4:56 PM on April 9, 2020 [11 favorites]
Good luck getting volunteers if that means they aren't getting paid. COBOL programmers can command a hefty price being called for situations like this or consulting age old systems that exist in more places than you can imagine.
This is also not limited to government systems that haven't been modernized, it's a known fact that many banks and financial institutions still rely on these mainframe systems for various reasons. It's not going away anytime soon, TBH.
posted by xtine at 5:01 PM on April 9, 2020 [20 favorites]
This is also not limited to government systems that haven't been modernized, it's a known fact that many banks and financial institutions still rely on these mainframe systems for various reasons. It's not going away anytime soon, TBH.
posted by xtine at 5:01 PM on April 9, 2020 [20 favorites]
I wonder if SABRE is still running on COBOL. Fun series of facts: American Airlines developed SABRE to handle their own reservations and then sold service to other companies. There were some years when SABRE made more money than American made flying airplanes.
There was discussion in the last thread about a debunked legend of a printer with its output directed to shredder. I just wanted to add tha this wasn't a problem even 30 years ago when I was doing main frame work. Getting paper outputs electronically was easy. Reading paper outputs digitally that were designed for paper forms was a bit more challenging.
Also Mainframes could/can crank out a lot of paper. The building I was in had a single printer that had multiple drawers for paper and each of the drawers accommodated two boxes of paper plus smaller drawers for things like coloured paper. The printer would automatically collate each print job, attach a cover sheet and then bundle the jobs by user if you printed more than one job in a set period of time. It was a full time job just keeping the thing supplied with paper. And another full time job distributing print jobs. I used to get internal mail delivered 4-5 times a day.
posted by Mitheral at 5:06 PM on April 9, 2020 [3 favorites]
There was discussion in the last thread about a debunked legend of a printer with its output directed to shredder. I just wanted to add tha this wasn't a problem even 30 years ago when I was doing main frame work. Getting paper outputs electronically was easy. Reading paper outputs digitally that were designed for paper forms was a bit more challenging.
Also Mainframes could/can crank out a lot of paper. The building I was in had a single printer that had multiple drawers for paper and each of the drawers accommodated two boxes of paper plus smaller drawers for things like coloured paper. The printer would automatically collate each print job, attach a cover sheet and then bundle the jobs by user if you printed more than one job in a set period of time. It was a full time job just keeping the thing supplied with paper. And another full time job distributing print jobs. I used to get internal mail delivered 4-5 times a day.
posted by Mitheral at 5:06 PM on April 9, 2020 [3 favorites]
(Just for reference in case future readers are interested, there was a sub-discussion on this woven thru part of this US politics/coronavirus thread earlier)
posted by LobsterMitten at 5:19 PM on April 9, 2020
posted by LobsterMitten at 5:19 PM on April 9, 2020
The last job I had was with a software startup. Our product was a realtime adjudication system for doctors’ offices that spoke directly to insurers’ systems. That last part was why we had a very-well-paid COBOL guy on staff.
posted by Thorzdad at 5:26 PM on April 9, 2020 [5 favorites]
posted by Thorzdad at 5:26 PM on April 9, 2020 [5 favorites]
My partner said the way COBOL is designed like it is is so that your manager who is not a programmer but is an MBA can (at least in theory) read the code and discover if you're taking the piss or not.
I never learned the damn thing and don't know how it's changed, but in its original scope each line was meant to be punched on a card and character position was significant.
posted by seanmpuckett at 5:40 PM on April 9, 2020 [2 favorites]
I never learned the damn thing and don't know how it's changed, but in its original scope each line was meant to be punched on a card and character position was significant.
posted by seanmpuckett at 5:40 PM on April 9, 2020 [2 favorites]
FORTRAN was like that too - anything in the first column (traditionally a C) made the line a comment, 2-5 are the numeric label, for a branch target, column 6 (traditionally a +) makes the rest of the line a continuation of the previous one, and actual code couldn't start till column 7.
And heavy on the "in theory" about MBAs reading COBOL - but the theory was certainly an important selling point. Plus ça change...
posted by bcd at 6:03 PM on April 9, 2020 [2 favorites]
And heavy on the "in theory" about MBAs reading COBOL - but the theory was certainly an important selling point. Plus ça change...
posted by bcd at 6:03 PM on April 9, 2020 [2 favorites]
My father in law just retired and was a COBOL programmer for a government agency. We were wondering if they would take the chance to switch things up, but nope, they're going to replace him with a new COBOL programmer.
posted by damayanti at 6:04 PM on April 9, 2020 [1 favorite]
posted by damayanti at 6:04 PM on April 9, 2020 [1 favorite]
Unfortunately, there isn't an easy solution for most of these legacy systems. As expensive as hiring an experienced COBOL archaeologist to sift through the layers of code to add a patch or fix a bug is, it's usually cheaper than the 2 most common alternatives:
1. Buying into a new proprietary system from a commercial vendor. Will the setup and transition happen on time? (No) Will your users get the training that was promised? (Ha ha ha ha!) Will the license fees be similar or consistent or even based on the same metrics (like user seats vs. #of customers in database vs. sales volume...) once the initial contract is up? (Hey, I'm just a sales guy not a fortune teller!) How long can you expect first-party support, bugfixes and upgrades before the company moves on to a new product, new business model, or just goes away.
2. Migrating to a custom solution based on Common Standards, using tools and software that is Open Source or at least supported by multiple vendors. This usually seems to be the smartest approach, but it all hinges on the skills and capabilities of whomever is managing the project - and especially their ability to communicate the needs for a proper spec and documentation. I'm sure there are success stories out there that I'm not aware of, but every time I've seen this tried, it has been doomed by a combination of hubris, naivete and laziness.
If you think 40 year old COBOL is hard to maintain, try looking under the hood of a large scale IT project that ran over time and budget (...and which ones didn't) and was "saved" by throwing a couple dozen contract coders on at the end who were brought up in the era of webcode, idiosyncratic scripting languages and *shudder* "frameworks". Every time you need a new feature or bug fix, be assured that the newest person to touch the code will end up re-writing large chunks of it from scratch - because it's "easier" for them, and because they're billing you by the hour.
So, summoning the COBOL greybeards every time you need a patch starts to seem less of a lazy hack, and more like a good, practical use of resources.
posted by Anoplura at 6:05 PM on April 9, 2020 [41 favorites]
1. Buying into a new proprietary system from a commercial vendor. Will the setup and transition happen on time? (No) Will your users get the training that was promised? (Ha ha ha ha!) Will the license fees be similar or consistent or even based on the same metrics (like user seats vs. #of customers in database vs. sales volume...) once the initial contract is up? (Hey, I'm just a sales guy not a fortune teller!) How long can you expect first-party support, bugfixes and upgrades before the company moves on to a new product, new business model, or just goes away.
2. Migrating to a custom solution based on Common Standards, using tools and software that is Open Source or at least supported by multiple vendors. This usually seems to be the smartest approach, but it all hinges on the skills and capabilities of whomever is managing the project - and especially their ability to communicate the needs for a proper spec and documentation. I'm sure there are success stories out there that I'm not aware of, but every time I've seen this tried, it has been doomed by a combination of hubris, naivete and laziness.
If you think 40 year old COBOL is hard to maintain, try looking under the hood of a large scale IT project that ran over time and budget (...and which ones didn't) and was "saved" by throwing a couple dozen contract coders on at the end who were brought up in the era of webcode, idiosyncratic scripting languages and *shudder* "frameworks". Every time you need a new feature or bug fix, be assured that the newest person to touch the code will end up re-writing large chunks of it from scratch - because it's "easier" for them, and because they're billing you by the hour.
So, summoning the COBOL greybeards every time you need a patch starts to seem less of a lazy hack, and more like a good, practical use of resources.
posted by Anoplura at 6:05 PM on April 9, 2020 [41 favorites]
USDA is upgrading their COBOL version starting in the fall and it's gonna take a year to do so.
posted by 922257033c4a0f3cecdbd819a46d626999d1af4a at 6:17 PM on April 9, 2020 [1 favorite]
posted by 922257033c4a0f3cecdbd819a46d626999d1af4a at 6:17 PM on April 9, 2020 [1 favorite]
I know of at least one major bank with a program to train up a new generation of COBOL programmers internally. I doubt they're alone.
posted by paper chromatographologist at 6:34 PM on April 9, 2020 [1 favorite]
posted by paper chromatographologist at 6:34 PM on April 9, 2020 [1 favorite]
COBOL is one of the few well-known languages I've never actually programmed in, but for all of its idiosyncrasies--some of which, like column-significance were typical of its era--it has its merits. But I'm familiar enough with it to venture some thoughts.
COBOL is a very I/O-oriented language. You can declare report sections with "pictures" and deal with input similarly. Database records can be mapped to structured data in ways that modern database frameworks and DSLs rarely improve upon. Arithmetic is pretty primitive and, traditionally, in decimal. But you know, decimal arithmetic is pretty much what you want for ledgers, and since a typical program is I/O bound it really doesn't matter how fast arithmetic is. For "data processing" purposes it's pretty fit for purpose.
I mean, to a CS nerd like me the language is an odd duck. But it directly addressed the practical issues it was concerned with instead of creating abstraction layers that might or might not facilitate library- or framework-level support for them. C++ (until this last decade at least) was a story of promise and disappointment. Java was going to be the solution but nobody will admit to remembering that anymore. C#/.NET was going to be a better Java, and actually was in many ways, but never fulfilled its promise either.
What I'm saying is that COBOL had practical and humble goals that it satisfactorily met. Every language/platform/framework can be pushed beyond its limits, and eventually all of them are. But COBOL was actually pretty good at doing what it was designed to do, which is something you couldn't say of C++ until it was 30 years old.
posted by sjswitzer at 6:46 PM on April 9, 2020 [24 favorites]
COBOL is a very I/O-oriented language. You can declare report sections with "pictures" and deal with input similarly. Database records can be mapped to structured data in ways that modern database frameworks and DSLs rarely improve upon. Arithmetic is pretty primitive and, traditionally, in decimal. But you know, decimal arithmetic is pretty much what you want for ledgers, and since a typical program is I/O bound it really doesn't matter how fast arithmetic is. For "data processing" purposes it's pretty fit for purpose.
I mean, to a CS nerd like me the language is an odd duck. But it directly addressed the practical issues it was concerned with instead of creating abstraction layers that might or might not facilitate library- or framework-level support for them. C++ (until this last decade at least) was a story of promise and disappointment. Java was going to be the solution but nobody will admit to remembering that anymore. C#/.NET was going to be a better Java, and actually was in many ways, but never fulfilled its promise either.
What I'm saying is that COBOL had practical and humble goals that it satisfactorily met. Every language/platform/framework can be pushed beyond its limits, and eventually all of them are. But COBOL was actually pretty good at doing what it was designed to do, which is something you couldn't say of C++ until it was 30 years old.
posted by sjswitzer at 6:46 PM on April 9, 2020 [24 favorites]
Why New Jersey’s Unemployment Insurance System Uses a 60-Year-Old Programming Language - Elena Botella for Slate
posted by ZeusHumms at 6:51 PM on April 9, 2020 [1 favorite]
posted by ZeusHumms at 6:51 PM on April 9, 2020 [1 favorite]
Corona Only Breaks Old Languages?
because the other languages were already broken...
posted by Huffy Puffy at 6:59 PM on April 9, 2020 [6 favorites]
because the other languages were already broken...
posted by Huffy Puffy at 6:59 PM on April 9, 2020 [6 favorites]
Grace Hopper - The Mother of Cobol (i-programmer.info, 2018)
posted by dmh at 7:13 PM on April 9, 2020 [11 favorites]
Rear Admiral Grace Brewster Murray Hopper was responsible not only for the development of the Cobol language but also for the continuous pressure within the industry to make computers and computing more accessible and to bring the research and career interests of women in computing to the forefront.Captain Grace M. Hopper: The Mother of COBOL (scan from InfoWorld, 1981)
posted by dmh at 7:13 PM on April 9, 2020 [11 favorites]
Canada would like to offer you this object lesson on the dangers of trying to replace your COBOL-based data processing systems. Current cost estimates for the whole mess are running around $2.6 Billion.
I currently owe my employer $3200 because Phoenix double issued all my retroactive contract adjustments back in January. No idea when they are going to ask for it back, but I put it in its own little account at my credit union so I wouldn't accidentally spend it. Won't do much to help with the $2.6B bill, though, even when they recover it.
posted by jacquilynne at 7:30 PM on April 9, 2020 [23 favorites]
I currently owe my employer $3200 because Phoenix double issued all my retroactive contract adjustments back in January. No idea when they are going to ask for it back, but I put it in its own little account at my credit union so I wouldn't accidentally spend it. Won't do much to help with the $2.6B bill, though, even when they recover it.
posted by jacquilynne at 7:30 PM on April 9, 2020 [23 favorites]
I support a minor but fairly impactful enterprise product. It had about a 12 year life from first release to last really big release. Another 12 years of waning development and winding down support, with a couple more meaningful releases to come yet. And I wager that it will be another 12 years after the last release before the last site stops running it. The modern operating systems on our computers are 25-30 years old, with direct antecedents going back 40 or 50 years, and both Word and Excel are damn near 40 years old and nobody has bothered to file away an obituary for those.
Big kid, grown up software has a lifetime spanning lifecycle already.
That said, I read a couple pages of COBOL and my life is too short to want anything to do with That stuff.
posted by wotsac at 9:42 PM on April 9, 2020 [2 favorites]
Big kid, grown up software has a lifetime spanning lifecycle already.
That said, I read a couple pages of COBOL and my life is too short to want anything to do with That stuff.
posted by wotsac at 9:42 PM on April 9, 2020 [2 favorites]
The media going on about COBOL = old, forget that Java is already 25 years old.
posted by PenDevil at 10:33 PM on April 9, 2020 [9 favorites]
posted by PenDevil at 10:33 PM on April 9, 2020 [9 favorites]
One of the government agencies I work with replaced their old financial system a few years ago. The result could charitably be described as a total ongoing fuck show. The company who sold it to them made a lot of money though and I guess that's what counts?
posted by fshgrl at 11:57 PM on April 9, 2020 [2 favorites]
posted by fshgrl at 11:57 PM on April 9, 2020 [2 favorites]
I haven’t worked in the airline software world for about 7 years but 7 years ago my employer at the time (a SABRE competitor) still had a few COBOL lifers on staff for the half-dozen clients that simply weren’t able/ready to migrate to the new .Net/MS SQL platform. So I’m pretty sure a significant number of SABRE’s customers are relying on COBOL-heavy environments too.
The bigger challenge with COBOL in my experience is less about securing programmers and more about the increasingly smaller list of hardware vendors who are willing to support it.
posted by Doleful Creature at 12:05 AM on April 10, 2020
The bigger challenge with COBOL in my experience is less about securing programmers and more about the increasingly smaller list of hardware vendors who are willing to support it.
posted by Doleful Creature at 12:05 AM on April 10, 2020
Correction: Assuming this recent airliners.net forum post is credible, I was wrong about SABRE using COBOL. Apparently it’s a “mix of SABREtalk*, C, and C++
*reportedly a variant of PL/1
posted by Doleful Creature at 12:12 AM on April 10, 2020
*reportedly a variant of PL/1
posted by Doleful Creature at 12:12 AM on April 10, 2020
I would happily come out of retirement to support either Cobol or PL/1 as I used both for most of my working life. (40+ years)
It would have to be remote though as I live in the UK and I am a registered invalid with nerve damage to my legs.
posted by Burn_IT at 4:39 AM on April 10, 2020 [3 favorites]
It would have to be remote though as I live in the UK and I am a registered invalid with nerve damage to my legs.
posted by Burn_IT at 4:39 AM on April 10, 2020 [3 favorites]
People still have paid full time jobs maintaining COBOL systems.
Very well paid COBOL jobs in most cases.
posted by DarlingBri at 5:40 AM on April 10, 2020 [1 favorite]
Very well paid COBOL jobs in most cases.
posted by DarlingBri at 5:40 AM on April 10, 2020 [1 favorite]
USDA is upgrading their COBOL version starting in the fall and it's gonna take a year to do so.I think that's a different part of USDA than I work for, but this will almost certainly be a disaster. We're in the middle of a slow-motion train-wreck of an IT consolidation project that probably makes sense for 95% of the Department. Unfortunately, I work for the other 5%. We were told this week that the consolidation is proceeding, nobody knows how much it's going to cost, and we should just budget as we did last year because the goal is for it to be "cost neutral". And, oh yeah, they're probably going to steal all of our IT specialists in year 2, even those who work only on research projects and have never done end-user support.
My team doesn't use COBOL, but we have miles of Fortran, and it's lovely. Clean and fast!
posted by wintermind at 7:05 AM on April 10, 2020 [2 favorites]
My partner said the way COBOL is designed like it is is so that your manager who is not a programmer but is an MBA can (at least in theory) read the code and discover if you're taking the piss or not.
Haha no. Cobol is totally unreadable and unfollowable nonsense.
See here. All that is required, including the self-input line numbers.
The one advantage I've heard (from programmers and business people alike) why insurers and banks don't ditch COBOL is that it handles numbers to a nearly infinite decimals, ie: no rounding unless you personally define it. So that 'Office Space' thing is real, and used by banks and insurers.
posted by The_Vegetables at 7:09 AM on April 10, 2020 [2 favorites]
Haha no. Cobol is totally unreadable and unfollowable nonsense.
See here. All that is required, including the self-input line numbers.
The one advantage I've heard (from programmers and business people alike) why insurers and banks don't ditch COBOL is that it handles numbers to a nearly infinite decimals, ie: no rounding unless you personally define it. So that 'Office Space' thing is real, and used by banks and insurers.
posted by The_Vegetables at 7:09 AM on April 10, 2020 [2 favorites]
Should we be laughing at COBOL, knowing what Metafilter runs on? <3
posted by Melismata at 7:22 AM on April 10, 2020 [10 favorites]
posted by Melismata at 7:22 AM on April 10, 2020 [10 favorites]
As expensive as hiring an experienced COBOL archaeologist to sift through the layers of code to add a patch or fix a bug is, it's usually cheaper than the 2 most common alternatives:
It’s a splendid illustration of the axiom that states any question beginning, “Why don’t they...?” can be answered, “Money.”
posted by ricochet biscuit at 7:26 AM on April 10, 2020 [2 favorites]
It’s a splendid illustration of the axiom that states any question beginning, “Why don’t they...?” can be answered, “Money.”
posted by ricochet biscuit at 7:26 AM on April 10, 2020 [2 favorites]
More correctly, COBOL supports fixed-point arithmetic, which is usually what you want when you desire results in dollars-and-cents. So my NET_PAYCHECK_AMT PIC S9(7)V9(2) variable will always have two decimal places. Rounding rules can be explicitly applied to any calculation, so the result is unambiguous. (Important because the way amounts are rounded is often specified by legal contract.)
COBOL, in its defense, actually wasn't bad as a first attempt at a COmmon Business Oriented Language. But complexity has a way of expanding to fill the available computational space, so as demands increased (direct input from terminals, database interfaces, error trapping), the syntax never could quite keep up with the new requirements.
posted by SPrintF at 7:26 AM on April 10, 2020 [2 favorites]
COBOL, in its defense, actually wasn't bad as a first attempt at a COmmon Business Oriented Language. But complexity has a way of expanding to fill the available computational space, so as demands increased (direct input from terminals, database interfaces, error trapping), the syntax never could quite keep up with the new requirements.
posted by SPrintF at 7:26 AM on April 10, 2020 [2 favorites]
COBOL is like learning to play first base. It’s not that hard.
posted by Huffy Puffy at 7:39 AM on April 10, 2020 [1 favorite]
posted by Huffy Puffy at 7:39 AM on April 10, 2020 [1 favorite]
From Hacker News
A quick skim of the postings is finding that what they're really screaming for is somebody who's been familiar with their particular pile of toxic waste for at least four years at minimum. They're asking for people with 4+ years of COBOL, and JCL (ok, fine), and DB2, and VSAM, and "SDLC" (proto-waterfall), and, oh yeah, a four-year degree.
Heck, there's a posting asking for somebody to draw diagrams and document their existing processes, and for that they expect you to have 10 years of COBOL experience.
As usual, the press has the wrong idea of what is required. They don't need COBOL programmers; they need people who understand a range of obscure technologies, put together over many years and undocumented in large part, with fragile interdepencies.
Back in the 1970s I worked at a very large bank that used COBOL on some systems. They had one routine in one large program that was simply a jump to the routine, and a return. They couldn't remove this empty code, because if they did, something broke. Tracking this bug down wasn't as important as fixing other bugs, so they left an empty routine in place, with a comment.
That sort of "black magic" is all over every and any large production system. Maybe not as egregious as this example, but similar crazy solutions exist all over large systems.
posted by blob at 7:47 AM on April 10, 2020 [9 favorites]
A quick skim of the postings is finding that what they're really screaming for is somebody who's been familiar with their particular pile of toxic waste for at least four years at minimum. They're asking for people with 4+ years of COBOL, and JCL (ok, fine), and DB2, and VSAM, and "SDLC" (proto-waterfall), and, oh yeah, a four-year degree.
Heck, there's a posting asking for somebody to draw diagrams and document their existing processes, and for that they expect you to have 10 years of COBOL experience.
As usual, the press has the wrong idea of what is required. They don't need COBOL programmers; they need people who understand a range of obscure technologies, put together over many years and undocumented in large part, with fragile interdepencies.
Back in the 1970s I worked at a very large bank that used COBOL on some systems. They had one routine in one large program that was simply a jump to the routine, and a return. They couldn't remove this empty code, because if they did, something broke. Tracking this bug down wasn't as important as fixing other bugs, so they left an empty routine in place, with a comment.
That sort of "black magic" is all over every and any large production system. Maybe not as egregious as this example, but similar crazy solutions exist all over large systems.
posted by blob at 7:47 AM on April 10, 2020 [9 favorites]
I know of at least one major bank with a program to train up a new generation of COBOL programmers internally.
How hard is COBOL to learn? Is it easier if you have experience with other languages?
posted by chimpsonfilm at 7:47 AM on April 10, 2020
How hard is COBOL to learn? Is it easier if you have experience with other languages?
posted by chimpsonfilm at 7:47 AM on April 10, 2020
"The bigger challenge with COBOL in my experience is less about securing programmers and more about the increasingly smaller list of hardware vendors who are willing to support it."
Surely it should be possible to run old COBOL in some kind of virtualized hardware if necessary?
I'm surprised by how binary these decisions sound. Aren't there options between "keep the old system on life support with minimal bugfixes" and "throw it out and replace it with something new"? Virtualization? Rewriting old systems piece by piece?
I know nothing about this stuff, so I'm sure I'm just naive.
posted by bfields at 7:49 AM on April 10, 2020 [1 favorite]
Surely it should be possible to run old COBOL in some kind of virtualized hardware if necessary?
I'm surprised by how binary these decisions sound. Aren't there options between "keep the old system on life support with minimal bugfixes" and "throw it out and replace it with something new"? Virtualization? Rewriting old systems piece by piece?
I know nothing about this stuff, so I'm sure I'm just naive.
posted by bfields at 7:49 AM on April 10, 2020 [1 favorite]
Virtualization? Rewriting old systems piece by piece?
Most older business software has to be rewritten to be virtualized. The company I work for ditched Cobol 12 years ago, I don't recall it being that expensive or difficult (with the caveat that the majority of the projects we work on are moderately expensive) and some take a long time.
Just like anywhere else though, modernization projects have to be cheerleaded, funded, and maintained, and is subject to political whims, especially for state projects.
posted by The_Vegetables at 8:26 AM on April 10, 2020 [1 favorite]
Most older business software has to be rewritten to be virtualized. The company I work for ditched Cobol 12 years ago, I don't recall it being that expensive or difficult (with the caveat that the majority of the projects we work on are moderately expensive) and some take a long time.
Just like anywhere else though, modernization projects have to be cheerleaded, funded, and maintained, and is subject to political whims, especially for state projects.
posted by The_Vegetables at 8:26 AM on April 10, 2020 [1 favorite]
I currently owe my employer $3200
At one point I was notionally keeping track of the amount I was owed by a count of the number of Hyundai Accents it could buy. At the worst point it got to three and a half. Actings and on-call payments. /Phoenixbitching
OTOH, the cobol gurus at the tax agency just did some incredible magic over the past two weeks with the CERB relief payments. Unbelievable how quickly that's come together and how well that's worked. People apply and get their deposits a day later. Incredible.
posted by bonehead at 8:35 AM on April 10, 2020 [5 favorites]
At one point I was notionally keeping track of the amount I was owed by a count of the number of Hyundai Accents it could buy. At the worst point it got to three and a half. Actings and on-call payments. /Phoenixbitching
OTOH, the cobol gurus at the tax agency just did some incredible magic over the past two weeks with the CERB relief payments. Unbelievable how quickly that's come together and how well that's worked. People apply and get their deposits a day later. Incredible.
posted by bonehead at 8:35 AM on April 10, 2020 [5 favorites]
Most older business software has to be rewritten to be virtualized.
But, why? I can apparently run most old video game ROMs unchanged under emulation on new hardware, why can't I do the same for old business software?
Is the market just too small to make it anyone's worth to write a perfect emulator for a lot of old mainframe hardware? Are there intellectual property issues? Performance obstacles?
posted by bfields at 8:38 AM on April 10, 2020
But, why? I can apparently run most old video game ROMs unchanged under emulation on new hardware, why can't I do the same for old business software?
Is the market just too small to make it anyone's worth to write a perfect emulator for a lot of old mainframe hardware? Are there intellectual property issues? Performance obstacles?
posted by bfields at 8:38 AM on April 10, 2020
“I know of at least one major bank with a program to train up a new generation of COBOL programmers internally. I doubt they're alone.”
My alma mater (a Canadian community college) graduates hundreds of programmers every year, and COBOL is part of the core curriculum because it’s one of the skills that the government and banks have consistently asked the college to deliver.
posted by Secret Sparrow at 9:21 AM on April 10, 2020 [1 favorite]
My alma mater (a Canadian community college) graduates hundreds of programmers every year, and COBOL is part of the core curriculum because it’s one of the skills that the government and banks have consistently asked the college to deliver.
posted by Secret Sparrow at 9:21 AM on April 10, 2020 [1 favorite]
But, why? I can apparently run most old video game ROMs unchanged under emulation on new hardware, why can't I do the same for old business software?
Because it's bigger, more complex (in term of interacting systems), and required to be robust than video games. The article says that 362,000 people have used the NJ unemployment system in the past few weeks.
posted by The_Vegetables at 9:38 AM on April 10, 2020 [1 favorite]
Because it's bigger, more complex (in term of interacting systems), and required to be robust than video games. The article says that 362,000 people have used the NJ unemployment system in the past few weeks.
posted by The_Vegetables at 9:38 AM on April 10, 2020 [1 favorite]
IBM will happily sell you a POWER9 system that will run COBOL (or RPG or FORTRAN) programs and their many dependencies in a VM. Real nice system, though. It'll even turn your old COBOL shit into a web site capable of handling traffic that would give Google pause. Thing is, if you have to ask what it costs, you can't afford it.
(Other people still sell similar solutions, but nobody in the volume IBM still does)
posted by wierdo at 10:14 AM on April 10, 2020 [2 favorites]
(Other people still sell similar solutions, but nobody in the volume IBM still does)
posted by wierdo at 10:14 AM on April 10, 2020 [2 favorites]
But, why? I can apparently run most old video game ROMs unchanged under emulation on new hardware, why can't I do the same for old business software?
The fact that you can run those ROMs represents thousands of hours of volunteer labor on open-source emulators.
posted by firechicago at 10:16 AM on April 10, 2020 [8 favorites]
The fact that you can run those ROMs represents thousands of hours of volunteer labor on open-source emulators.
posted by firechicago at 10:16 AM on April 10, 2020 [8 favorites]
There are COBOL implementations for pretty much any computer you can find but the quality and level of support might not be what you need for software infrastructure that's been running for 40 years and might need to run for 30 more. Who you gonna call? Well... IBM is still a safe bet. Anyone else?
The program is not the whole picture by any means. These systems consist of software, I/O formats and devices, a live database, a redundancy and backup architecture, transactionality requirements, etc. that are served by a combination of the operating system, the database and hardware architecture.
Versions of COBOL are not entirely compatible, so you'll need to make some changes to your code and might even need to understand it.
The converted software will need to continue dealing with the data the old software was managing... seamlessly and probably without interruption. Old IBM systems store character data in EBCDIC rather than ASCII or any variety of Unicode. So you'll either have to deal with EBCDIC on a non-IBM machine or bulk-convert your database. Oh, BTW, there is no complete 1-1 mapping between EBCDIC and ASCII.
The upshot of this is once IBM, always IBM. IBM will gladly sell you modern systems that can meet all of these requirements and support you well into the future. That's a valuable service and is priced accordingly.
posted by sjswitzer at 10:45 AM on April 10, 2020 [10 favorites]
The program is not the whole picture by any means. These systems consist of software, I/O formats and devices, a live database, a redundancy and backup architecture, transactionality requirements, etc. that are served by a combination of the operating system, the database and hardware architecture.
Versions of COBOL are not entirely compatible, so you'll need to make some changes to your code and might even need to understand it.
The converted software will need to continue dealing with the data the old software was managing... seamlessly and probably without interruption. Old IBM systems store character data in EBCDIC rather than ASCII or any variety of Unicode. So you'll either have to deal with EBCDIC on a non-IBM machine or bulk-convert your database. Oh, BTW, there is no complete 1-1 mapping between EBCDIC and ASCII.
The upshot of this is once IBM, always IBM. IBM will gladly sell you modern systems that can meet all of these requirements and support you well into the future. That's a valuable service and is priced accordingly.
posted by sjswitzer at 10:45 AM on April 10, 2020 [10 favorites]
Also, all the reasonable ways of doing a conversion well spend a lot of money early in the process - you need programmers in the old system, and in the new system, and the people who can explain how the system is actually used, and good managers who can look at incompatibilities between the three and adjudicate. But we haaaaate spending money for maintenance or reliability upfront. Haaaaate it. So we go, over and over, for `fix it in post'.
posted by clew at 10:57 AM on April 10, 2020 [6 favorites]
posted by clew at 10:57 AM on April 10, 2020 [6 favorites]
Cobol is totally unreadable and unfollowable nonsense.
You've never seen MUMPS, have you? Now that's some sanity-melting nonsense.
posted by Mr. Bad Example at 11:23 AM on April 10, 2020 [7 favorites]
You've never seen MUMPS, have you? Now that's some sanity-melting nonsense.
posted by Mr. Bad Example at 11:23 AM on April 10, 2020 [7 favorites]
The Unisys mainframes where I work are virtualized and run on re-branded x86 Dell servers.
posted by tracknode at 11:29 AM on April 10, 2020
posted by tracknode at 11:29 AM on April 10, 2020
The Unisys mainframes where I work are virtualized and run on re-branded x86 Dell servers.
Which is a neat trick because those are 36-bit one's complement machines!
posted by sjswitzer at 11:34 AM on April 10, 2020
Which is a neat trick because those are 36-bit one's complement machines!
posted by sjswitzer at 11:34 AM on April 10, 2020
COBOL -- maybe not the language they need, but certainly the one they deserve ;)
Never again...
posted by twidget at 11:38 AM on April 10, 2020
Never again...
posted by twidget at 11:38 AM on April 10, 2020
run on re-branded x86 Dell servers
To be clear here, just because it uses x86 processors doesn't mean it's not a "mainframe." The thing that distinguishes a mainframe from the sort of computer you'd buy at Best Buy or even the computers filling up racks in a Google or Amazon datacenter is a sophisticated interconnect that supports SMP, clustering and redundancy. Those are still big and expensive machines.
Not to say that you couldn't run it on a desktop and probably would in development scenarios, but the hardware you'd deploy on would still be a "mainframe" in every meaningful sense.
(He says, knowing nothing about this particular system.)
posted by sjswitzer at 11:54 AM on April 10, 2020 [2 favorites]
To be clear here, just because it uses x86 processors doesn't mean it's not a "mainframe." The thing that distinguishes a mainframe from the sort of computer you'd buy at Best Buy or even the computers filling up racks in a Google or Amazon datacenter is a sophisticated interconnect that supports SMP, clustering and redundancy. Those are still big and expensive machines.
Not to say that you couldn't run it on a desktop and probably would in development scenarios, but the hardware you'd deploy on would still be a "mainframe" in every meaningful sense.
(He says, knowing nothing about this particular system.)
posted by sjswitzer at 11:54 AM on April 10, 2020 [2 favorites]
I've seen different studies that say that somewhere between 55 and 99% of all projects to transition to an ERP go over time and budget. That's what you're accounting for if you decide to leave the legacy system behind.
posted by tofu_crouton at 11:56 AM on April 10, 2020 [5 favorites]
posted by tofu_crouton at 11:56 AM on April 10, 2020 [5 favorites]
For a while, I was hoping that perl would be the new COBOL so that I could have some measure of job security (or at least job opportunity) in my old age. Unfortunately, instead of processing trillions of dollars in transactions for banks, governments, and airlines, perl seems to be mainly used to power Web 1.0 bulletin boards about wrestling or Neopets or Blink-182 or whatever.
posted by mhum at 11:59 AM on April 10, 2020 [9 favorites]
posted by mhum at 11:59 AM on April 10, 2020 [9 favorites]
Yeah, there are days I regret not sticking with the RPG thing. Sure, you can lay me off. Just hope there aren't any new kinds of taxes and don't plan to make any changes to your business processes between now and whenever that new system the consultants have been trying and failing to make work for a decade finally does make it into operation.
posted by wierdo at 12:07 PM on April 10, 2020 [1 favorite]
posted by wierdo at 12:07 PM on April 10, 2020 [1 favorite]
RPG is pretty cool in its peculiar way. It fits somewhere between patch-panel-programmed tabulator machines (which it was directly inspired by) and awk. Closer to the former, obviously.
posted by sjswitzer at 12:13 PM on April 10, 2020 [1 favorite]
posted by sjswitzer at 12:13 PM on April 10, 2020 [1 favorite]
I've seen different studies that say that somewhere between 55 and 99% of all projects to transition to an ERP go over time and budget.
Maybe so, but remember that 'quality' or some idea of 'customer or user satisfaction' is not a metric considered when those two (on-time and on-budget) are discussed by the management, and think of a world where the primary driver is to make them on-time and on-budget.
It would be worse in literally every way, except for the person who guesses at the time and budget (in my dev group that would be me). I personally am willing to take the heat for my guesses being wrong, and apparently so are about 45% of my peers. I'm scared of the other 55%.
posted by The_Vegetables at 12:20 PM on April 10, 2020 [1 favorite]
Maybe so, but remember that 'quality' or some idea of 'customer or user satisfaction' is not a metric considered when those two (on-time and on-budget) are discussed by the management, and think of a world where the primary driver is to make them on-time and on-budget.
It would be worse in literally every way, except for the person who guesses at the time and budget (in my dev group that would be me). I personally am willing to take the heat for my guesses being wrong, and apparently so are about 45% of my peers. I'm scared of the other 55%.
posted by The_Vegetables at 12:20 PM on April 10, 2020 [1 favorite]
And you might still have to use some sort of web-hosted windows desktop to have an old enough version of internet explorer to access your "modern" ERP system.
posted by sjswitzer at 12:47 PM on April 10, 2020 [1 favorite]
posted by sjswitzer at 12:47 PM on April 10, 2020 [1 favorite]
→ Cobol is totally unreadable and unfollowable nonsense
so, worse than
The C# example is 9 lines of CamelCase fluff in a 10 line program. C++ … nah, I can't even.
COBOL's structured like a contract. It tells you what it is, what it needs, what it uses, and then finally the actual detail of what it does. For stable business processes you don't need whizzo stuff: you just need the lowest cost per transaction.
Well into Web 2.0 days a friend was managing a business unit of semi-retired mainframe types for a huge multinational. They were bidding on internal computing jobs and won a lot because so much of the code was mature and running on virtualized servers wherever they were needed, glued onto web front ends. The programmers and systems folks were working from home, spread out all over the world. It was good work and well paid. Sometimes it didn't even need lower case …
I still think the high priests of CS haven't forgiven COBOL and Fortran for letting the unwashed secular users run jobs on the CS-sacrosanct hardware. That, or the former CS postdocs are still mad at how much more they could have earned being COBOL programmers.
posted by scruss at 1:24 PM on April 10, 2020 [12 favorites]
so, worse than
namespace HelloWorld
{
class Program
{
static void Main(string[] args)
{
System.Console.WriteLine("Hello world!");
}
}
}
or
#include <iostream>
int main () {
std::cout << "Hello world!" << std::endl;
}
?The C# example is 9 lines of CamelCase fluff in a 10 line program. C++ … nah, I can't even.
COBOL's structured like a contract. It tells you what it is, what it needs, what it uses, and then finally the actual detail of what it does. For stable business processes you don't need whizzo stuff: you just need the lowest cost per transaction.
Well into Web 2.0 days a friend was managing a business unit of semi-retired mainframe types for a huge multinational. They were bidding on internal computing jobs and won a lot because so much of the code was mature and running on virtualized servers wherever they were needed, glued onto web front ends. The programmers and systems folks were working from home, spread out all over the world. It was good work and well paid. Sometimes it didn't even need lower case …
I still think the high priests of CS haven't forgiven COBOL and Fortran for letting the unwashed secular users run jobs on the CS-sacrosanct hardware. That, or the former CS postdocs are still mad at how much more they could have earned being COBOL programmers.
posted by scruss at 1:24 PM on April 10, 2020 [12 favorites]
To be clear here, just because it uses x86 processors doesn't mean it's not a "mainframe."
Absolutely, but it's the software that makes them special. The hardware itself is commodity and not all that beefy compared to other systems in our environment. In terms of IO, our mainframes share the same SAN and disk subsystems that our windows servers use. You still pay by the cycle though. However, I work for a university so we're a small site compared to banking or government institutions. Also, I should state that I'm no expert with these systems, I just work down the hall from the people who are.
posted by tracknode at 3:46 PM on April 10, 2020 [1 favorite]
Absolutely, but it's the software that makes them special. The hardware itself is commodity and not all that beefy compared to other systems in our environment. In terms of IO, our mainframes share the same SAN and disk subsystems that our windows servers use. You still pay by the cycle though. However, I work for a university so we're a small site compared to banking or government institutions. Also, I should state that I'm no expert with these systems, I just work down the hall from the people who are.
posted by tracknode at 3:46 PM on April 10, 2020 [1 favorite]
The most recent couple of generations of (IBM) POWER9 servers are closer than ever to being commodity hardware, especially at the very lowest end of the product stack, but they are still far from what you'd find in any PC server. Some of the "magic" gets stuck on expansion boards in HPC clusters, but that's the closest you'll find.
I mean, sure, they use SAS drives and PCIe these days, but that doesn't make them replaceable with commodity hardware.
That's not to say that big iron systems are the best choice for many (if any) entirely new projects, if for no other reason than it being a lot harder to find people who know what they are doing. I will say that I'd much prefer working with a System i running DB2 to Oracle or even MSSQL on a PC, even to this day. That kind of work is the bread and butter of mainframes and their ilk.
posted by wierdo at 4:31 PM on April 10, 2020 [2 favorites]
I mean, sure, they use SAS drives and PCIe these days, but that doesn't make them replaceable with commodity hardware.
That's not to say that big iron systems are the best choice for many (if any) entirely new projects, if for no other reason than it being a lot harder to find people who know what they are doing. I will say that I'd much prefer working with a System i running DB2 to Oracle or even MSSQL on a PC, even to this day. That kind of work is the bread and butter of mainframes and their ilk.
posted by wierdo at 4:31 PM on April 10, 2020 [2 favorites]
The thing that distinguishes a mainframe from the sort of computer you'd buy at Best Buy or even the computers filling up racks in a Google or Amazon datacenter is a sophisticated interconnect that supports SMP, clustering and redundancy
I was amazed when, fresh into college, I first heard about how IBM's zSystem (and others) compensate for CPU failures, by running every instruction in parallel and comparing the results. If there's a discrepancy, the instruction is retried, and if the discrepancy persists, the faulty CPU is taken offline. Any jobs that were running on the faulty CPU get migrated and can usually keep running without interruption, all without human intervention or special accommodations in user-level software.
I have heard more than one story about IBM engineers appearing on-site to replace faulted components, to the surprise of the customer, who up until that point hadn't realized anything was amiss. "What? No, the computer is running fine!" "With all due respect, sir, the computer disagrees."
posted by dmh at 5:04 AM on April 11, 2020 [6 favorites]
I was amazed when, fresh into college, I first heard about how IBM's zSystem (and others) compensate for CPU failures, by running every instruction in parallel and comparing the results. If there's a discrepancy, the instruction is retried, and if the discrepancy persists, the faulty CPU is taken offline. Any jobs that were running on the faulty CPU get migrated and can usually keep running without interruption, all without human intervention or special accommodations in user-level software.
I have heard more than one story about IBM engineers appearing on-site to replace faulted components, to the surprise of the customer, who up until that point hadn't realized anything was amiss. "What? No, the computer is running fine!" "With all due respect, sir, the computer disagrees."
posted by dmh at 5:04 AM on April 11, 2020 [6 favorites]
Three CPUs? Five? As many as it takes to hit the nines in the contract?
posted by clew at 9:47 AM on April 11, 2020
posted by clew at 9:47 AM on April 11, 2020
IBM will offer free COBOL training to address overloaded unemployment systems
posted by adept256 at 11:01 AM on April 11, 2020 [4 favorites]
posted by adept256 at 11:01 AM on April 11, 2020 [4 favorites]
You know, I've worked at places that had really bad python and javascript code. We're talking real spaghetti code here, where it takes you hours and hours just to implement some fairly simple changes, because you have to follow the most mindbendingly abstruse codepaths. Code that doesn't seem to have much rhyme or reason to it, because it was hacked over by successive generations of programmers who were incentivized to work as quickly as possible. This is all using "modern" languages that provide you with all sorts of ways to organize your code, if you bother to use them. Which they didn't. On top of that, these languages give you the freedom to follow vastly different coding strategies -- freedom these programmers generously availed themselves of, in a myriad of different ways.
Now, granted, COBOL doesn't seem to have many of the things that make it easy to create maintainable, well-organized code. But surely the code that's out there can't all be uniformly awful? Likewise, is the bad COBOL code really all that much worse than the frighteningly twisted python and javascript code I've had to deal with? Or is it possible that the COBOL code might actually be easier to follow by virtue of being a simpler language? Asking here because I genuinely don't know.
posted by panama joe at 5:10 PM on April 11, 2020 [2 favorites]
Now, granted, COBOL doesn't seem to have many of the things that make it easy to create maintainable, well-organized code. But surely the code that's out there can't all be uniformly awful? Likewise, is the bad COBOL code really all that much worse than the frighteningly twisted python and javascript code I've had to deal with? Or is it possible that the COBOL code might actually be easier to follow by virtue of being a simpler language? Asking here because I genuinely don't know.
posted by panama joe at 5:10 PM on April 11, 2020 [2 favorites]
North of 25 years ago now, I had the dubious pleasure of teaching C to existing programmers from one of the largest banks in North America. What made the pleasure most dubious was that each class they sent me was comprised about equally of COBOL and Assembler programmers.
The set of things each group knew was largely disjoint. Pointers? Sure, says the Assembler crew. Blank looks from the COBOL side of the room. Data structures? Of course, say the COBOL folks. We'll just stuff an offset in the index register say the Assembler boys. Pulling teeth all around.
But as a round about answer to panama joe's question. Yes, I'd say so. COBOL is certainly more of a "there's one right way to do it" language than any of the current dynamically typed/dispatched languages.
posted by bcd at 8:14 PM on April 11, 2020 [2 favorites]
The set of things each group knew was largely disjoint. Pointers? Sure, says the Assembler crew. Blank looks from the COBOL side of the room. Data structures? Of course, say the COBOL folks. We'll just stuff an offset in the index register say the Assembler boys. Pulling teeth all around.
But as a round about answer to panama joe's question. Yes, I'd say so. COBOL is certainly more of a "there's one right way to do it" language than any of the current dynamically typed/dispatched languages.
posted by bcd at 8:14 PM on April 11, 2020 [2 favorites]
Unemployment checks are being held up by a coding language almost nobody knows (Makena Kelly, The Verge)
A survey by The Verge found that at least 12 states still use COBOL in some capacity in their unemployment systems. Alaska, Connecticut, California, Iowa, Kansas, and Rhode Island all run on the aging language. According to a spokesperson from the Colorado Department of Labor and Employment, the state was actually only a month or two away from “migrating into a new environment and away from COBOL,” before the COVID-19 pandemic hit.posted by ZeusHumms at 5:49 AM on April 14, 2020
As the pandemic has millions out of work, these systems have become a barrier for the recently unemployed. The federal labor department reported 16.8 million unemployment claims were filed between March 15th and April 4th. That’s approximately 13 percent of the US’s workforce, outstripping even the height of the 2008 financial crash, where unemployment topped off at around 10 percent. As more stores and businesses shutter as a result of the pandemic, the US’s unemployment systems are experiencing an unprecedented amount of traffic and requests — and states don’t have the resources to maintain them.
Likewise, is the bad COBOL code really all that much worse than the frighteningly twisted python and javascript code I've had to deal with? Or is it possible that the COBOL code might actually be easier to follow by virtue of being a simpler language?
In my experience, the older code is more readable because:
* Procedural code generally has less pathways for confusion.
* You won't have the javascript nightmare of different frameworks or, worse, different versions of the same framework scattered throughout the codebase.
* If someone has been coding for a long time in the same language and/or codebase, there's 30 years of built up standards there. Not necessarily good standards, but standards. It's more consistent.
posted by tofu_crouton at 7:48 AM on April 14, 2020 [1 favorite]
In my experience, the older code is more readable because:
* Procedural code generally has less pathways for confusion.
* You won't have the javascript nightmare of different frameworks or, worse, different versions of the same framework scattered throughout the codebase.
* If someone has been coding for a long time in the same language and/or codebase, there's 30 years of built up standards there. Not necessarily good standards, but standards. It's more consistent.
posted by tofu_crouton at 7:48 AM on April 14, 2020 [1 favorite]
« Older The Virus Changed the Way We Internet | "It always seems impossible until it is done." Newer »
This thread has been archived and is closed to new comments
posted by adept256 at 4:43 PM on April 9, 2020 [2 favorites]