This reminds me of my favorite bug at my first programming job.
I worked on a system that processed benefits. We generated a lot of reports, one of which being the overage dependent report, listing all children that were over the age limit for that insurance plan.
We had one case where a newborn was showing up on that report, and for end users, there was no indication why. DOB was 20-something days ago, not 20-something years ago.
Lo and behold, the paperwork was somehow filed before the baby was born, and our date calculation method was returning an unsigned int. So the logic was resulting in an age of -1 (since DOB was prior coverage start date), which wrapped around to 255, which was obviously higher than whatever the max dependent age was.
I spent some time debugging insurance systems, and bugs exactly like that were everywhere. There are lots of date comparisons in those kinds of systems, and the order in which those events happen are almost always described to happen in a certain order, but there's tons of exceptions for every case.
As soon as it made the 1919=2019 mistake you know it's the ancient tale of 2 digit years being insufficient for dates. Wonder how systems determine if it should be 19XX or 20XX? Does it just default to the most recent year?
>WWE 2K20, a professional wrestling video game, also stopped working at midnight on 1 January 2020. Within 24 hours, the game’s developers, 2K, issued a downloadable fix.
I had heard that that was actually an ISO year issue. For example 2019-12-30 becomes 2020-12-30 if you mistakenly use IYYY-MM-DD instead of YYYY-MM-DD (which kind of makes sense IYYY shows up first in documentation).
Haha I should have known it would be even simpler than trying to estimate which date is reasonable. Interestingly Oracle just treats all XX as 20XX from my brief testing when doing to_date with 'YYMMDD' as the format.
If you use the TO_DATE function with the YY datetime format element, then the year returned always has the same first 2 digits as the current year. If you use the RR datetime format element instead, then the century of the return value varies according to the specified two-digit year and the last two digits of the current year.
That is:
If the specified two-digit year is 00 to 49, then:
* If the last two digits of the current year are 00 to 49, then the returned year has the same first two digits as the current year.
* If the last two digits of the current year are 50 to 99, then the first 2 digits of the returned year are 1 greater than the first 2 digits of the current year.
If the specified two-digit year is 50 to 99, then:
* If the last two digits of the current year are 00 to 49, then the first 2 digits of the returned year are 1 less than the first 2 digits of the current year.
* If the last two digits of the current year are 50 to 99, then the returned year has the same first two digits as the current year.
Interesting I've never had to dig into that, started working well after Y2K and the company I work at, as far as I see, properly converted any legacy XX years ages ago.
Is the switch date for RR fixed? Seems like there will be some weird breakage around 2050 as random places that have been skating by on using RR for old 2 digit years will be in trouble.
Having worked in the OCR world pretty much non stop for about a decade, i'd say its more just a case of there not being a lot of 101 yr olds, and there being a lot of 1 year olds; the system is likely over-optimized for switching the digits to 20 v's 19 as it's more probable that the individual is 1 than 101. Really, when the confidence is low, or the data seems exceptional, it should be flagged for a manual review.
I'm not sure what the user experience is, my assumption is that it could be the individual or a parent scanning this data, so unless it was attached to a parent profile before entering the data, it's hard to know who is doing the administration. Having said that, if your assumption was correct ("The person is doing their own administration"), i'd agree with you.
That makes sense, thanks for the context! I've been through Canadian immigration processes (I'm from Australia), but have no idea how the UK/EU process works.
Attaching a confidence / accuracy score to the data isn't a step that's commonly taken. However for qualifying user input this sounds like a great idea.
This is making some very kind assumptions about basic administrative staff. For people that work in our industry, a confidence/accuracy score is something that's usually well understood, and is generally treated as an assessment of a single item in context with the whole.
Once you get to the general public it's another matter entirely; for instance, look at how 538 was pilloried for their "30% chance to win the presidency for the current head of state," and how many times they've put out detailed information on how 30% =/= 0%. Your general person running review of the data (not the engineer hiding in the back office tuning the results, who should have access to the information) is going to treat most things above 50% as inherently true and anything below 50% as inherently false (within some margin of error, i.e. 50% +/- 5 points).
It would be better to go the 'dumb' machine route of setting up some standard escalation flags; which looks like it may be what happened with this (albeit in a bad way), "person self filing and < 18 years of age; raise flag." The system being bad at differentiating 1919 & 2019 and then sending out an automated response seems more like a bad UX problem at this juncture, since it's something that could have been avoided with a quiet internal flagging instead of a loud external one.
I would be there at the time the data is scanned, but you're totally right. I imagine it could be added to a "metadata" or "notes" type section of the submission so the reviewer has the context the decision engine was using.
I don't think it's due to OCR because although the Home Office's app does take a picture of the passport's ID page it then actually scans the passport's chip.
So I don't really understand how it can misinterpret the birth date unless someone really stupidly coded the system to keep only the last 2 digits of year and then try to guess what it really was later...
Ugh, this isn't as bad as some dental software I use all the time. "1/1/48" entered as birth date is displayed as "1/1/1948". Within minutes, that is automatically updated to "1/1/2048". Yes, this patient we're treating today will be born in 28 years...
I thought this part was interesting;
> It then skipped the face recognition section which is what it does with under-12s
I'm aware that the passport photo conditions for young children are more relaxed (under 1 don't need their eyes open, under 6 don't need a 'neutral' expression) but I didn't expec that they would just skip the whole under-12s photo part.
Face recognition has difficulty with under 12s because they’re still developing so quickly, given only a single reference photo from five years ago the accuracy of facial recognition is going to be terrible.
This is a tangential question, but I'm curious how well it tends to do with adults who undergo things like weight gain/loss, emergence or clearing of acne, minor-to-moderate injuries -- the sorts of natural change that can happen quickly and make a significant difference, but rarely fool humans.
Hardly surprising, though, since in general practical (i.e. non-lab) conditions it has trouble with over 12's too. FP still pretty high without additional constraints (e.g. photos on your timeline)
I was under the impression that most airlines require you to be at least 12 to do "unaccompanied minor"; anyone younger than that should be with both parents or they warrant special checks at the border anyway (such as letters of permission from the parents).
The US says children must be 5 to travel "unaccompanied" (i.e. supervised by airport staff to the plane etc). From around 12-14 they can go alone, depending on the airline.
I think it's a mismatch in terminology. My brother sent his three kids to us for a holiday. The 11 year old had to be "accompanied" by a stewardess. The 13 and 16 year olds were able to travel "unaccompanied", as in not under the supervision of the airline.
I worked for some time in a bank in France.
Some migrants does not know their real birth date, so their civil status can officially contain a date with zeroes, like `1999-00-00`.
On the contrary, the social insurance system for some reason decided to store this case as `1999-13-13`.
I discovered that in a very complex system that temporarily stored data in a SQL Server's `DATETIME` field, which ended up with people having NULL birth dates and made the whole system collapse.
The system is probably still running today by excluding those people from some fraud detection procedures.
In Denmark we use Jan 1st too. That turned out to be a problem because our personal identification numbers (CPR-numbers) contains the birthdate, and suddenly we started running out of them for Jan 1st some years.
The solution was to remove the checksum requirement (it was a weighted modulo 11 checksum), which is still sometimes causing problems with software that tries to verify it using the old rules. But for some reason that was thought to be better than adding another digit, maybe because it "only" requires every piece of software working with CPR-numbers in existence to be updated but not the data storage...
Ah yes personal identification numbers (BSN in Holland) don't 'spark joy' either.
In Holland every resident gets a unique BSN. So you can use it to uniquely identify a person.
Except if you also have to deal with asylum seekers, because they get a slightly different kind of number which will be reused after the asylum seeker is granted/denied residency.
It puzzles me why he never became a citizen. He moved to the UK in 1966, that's 54 years now, and it was even before they joined the EEC sometime in the 1970s.
Doing so before 1992 (or as late as 2001 if notification requirements were not complied with) would have lost him his Italian citizenship, unless an exception applied. I can understand being satisfied with a stable situation for most of that time, and with not wanting to handle naturalization paperwork in the last decades of life.
Good point. I somehow assumed EC citizens would be exempt from such laws but that doesn't necessarily seem to be the case even now [1], and must have been very different before the 1992 Maastricht treaty indeed.
It's possible to like living somewhere and be comfortable with permanent residency, but to have compelling reasons to retain the citizenship of your birth country. Many countries require that you give up the old citizenship when you get a new one.
In my case, Germany requires citizenship applicants to show proof that they have attempted to give up their original citizenship, unless it's another EU country, and that's only because the EU high court ruled that they have to let EU citizens retain their EU citizenships.
I love living in Germany, have learned the language, contribute a lot to it and will probably live here the rest of my life, but I'm an American and have no intention of giving that up.
A user below, hkt, has a comment complaining about the culture at the Home Office. It has been modded down, and fairly so, as this comment is not helpful in its current form. Nevertheless I feel it has an important point, so allow me to talk about the Home Office, which, after all, is the reason this 101-year-old was filing paperwork to begin with.
The Home Office presently maintains a "hostile environment" policy. Wikipedia summarizes its goal: "to make staying in the United Kingdom as difficult as possible for people without leave to remain". Here are some examples of how this works in practice.
Once upon a time the BBC did a little investigative journalism and discovered that some international students were faking their English proficiency exams (the TOEIC). The Home Office, quite reasonably, said that they'd do something about it. What they ended up doing specifically was make the environment as hostile as possible; they:
- looked at 58,458 test results
- declared that 34,000 of them were obviously definitely cheaters
- declared that 22,600 of them were probably cheaters
- declared only about 2000 as probably legit
- sent letters to the 34,000 students denouncing them as frauds and demanding they leave the country
Needless to say these numbers are completely bunk. Nevertheless thousands of the students were scared away, or detained, or deported.
An astute reader will note that the coverage I have specifically linked describes this as potentially "bigger than Windrush", a separate scandal in which refugees were admitted to the UK, decades later various records of their admission destroyed, and then the government demanded they demonstrate evidence they were allowed to be present in the country.
Now, personal anecdote time: I got a job in London the same day they voted for Brexit. I later found out that as family of a European citizen, the form to get a proper residence card was 97+ pages. This is basically as a way to make the environment more hostile. (Rather than deal with that and the economic uncertainties of Brexit, including its impact on the tech sector and general political uncertainty from subsequent governments, I instead moved to New York and literally doubled my salary, thanks in no small part to the low pound).
In summary: This particular story is just the latest insult in a parade of injuries. Fear not, though! The behavior of the Home Office has not gone unnoticed. For instance, the USCIS has recently been taking a page from the same book:
> The American Immigration Lawyers Association has collected [over 140] examples of allegedly "incomplete" forms: an 8-year-old child who listed "none" for employment history but left the dates of employment field blank. An applicant who entered names of three siblings, but the form has spaces for four.
Not to detract from the rest of the comment, which is broadly correct (speaking as somebody who actually had to go through those 97 pages to get indefinite leave to remain, pre-2016), but the description of Windrush as "refugees [who] were admitted to the UK" is just wrong.
The Windrush migrants were not refugees - they were explicitly invited by the UK government, through a policy that promoted migration from "old colonies" (particularly the Caribbeans), in order to kickstart a postwar British economy dramatically short of manpower. The policy was so lax that there were no controls of any sort; on landing, people were barely handed a piece of paper that certified they had disembarked from a certain ship on that day. For all intents and purposes, these migrants were then treated like British citizens, but were not issued passports or other documents unless they demanded them (which was often a complex procedure). This was not seen as unusual, in a country that famously has no national ID scheme and where identification is still largely achieved through utility bills.
The Home Office kept a copy of the landing records for 50 years, which helped whenever someone from the Windrush years had to make his or her position official. In 2010, the HO decided these records took up too much space and just destroyed them, without even trying to digitize them. This happened a few months after the Tory-LibDem coalition government was installed, in October 2010, with Theresa May as Home Secretary. Anybody from Windrush subsequently found themselves unable to prove their citizenship status, resulting in deportations, lack of health assistance, and massive discrimination.
For the record, Mrs. May was also the inventor of the "hostile environment" policy, which was supposed to deliver on the Tory manifesto promise of pushing net migration under 100,000 a year. That indicator was around 250,000 at the time. After 10 years of "hostile environment", it's now around 280,000.
> This is bizarre, blame the passport for only having the last two digits and not the computer reading it.
I mean, the passport only showing 2 digits is worth mentioning. But I doubt this 101-year-old got much choice in exactly what format his passport contained his DOB - so the software needs to handle it. So if in doubt (e.g. both possibilities yield an age younger than oldest person on the planet) the software should be asking to confirm the century.
If you want to get really fancy, check the issue date first. If this is before the supposed 20xx birth date, the birth date must actually be 19xx. People aren't issued passports before they are born.
I'm wondering if this is a case of idiotic government regulations meeting contractor requirements:
The government stupidly decides to only put 2 digits on the passport. They sign a contract with a company to write OCR software for reading passports. The contractor points out "hey! There's only 2 digits here!", but the customer doesn't have an answer because that's some other department and it's already done. So the contractor just implements the software as specified, assuming that being older than 100 is impossible, because the customer couldn't come up with a solution for this problem.
> I'm wondering if this is a case of idiotic government regulations
The MRZ is an ICAO standard so you’ll need to blame the UN for this.
> The government stupidly decides to only put 2 digits on the passport.
This passport was issued by a foreign government though.
> because the customer couldn't come up with a solution for this problem.
The app schedules an appointment with a human, presumably that human will be able to differentiate between a 1-year-old and a 101-year-old. Perhaps the problem never existed?
The home office phone rep fixed this in 30(!) minutes.
>The MRZ is an ICAO standard so you’ll need to blame the UN for this.
>This passport was issued by a foreign government though.
From my 20 seconds of Google research, plus your comment, it looks like the MRZ is something that various governments agreed on at some point, so the government that issued it isn't completely to blame, it's all of them that stupidly thought that either 1) no one over the age of 100 would ever have a passport or 2) there would never be a 21st century (as seemed to be the assumption through much of the 20th century with things like this; everyone always assumed the first two digits of years were always "19" and that anything starting with "18" or less was ancient history, and anything with "20" or greater was science fiction and would never happen in our lifetimes).
So my first comment is still correct, I just can't blame the UK government entirely for it, but since they're one of the main members of the UN (being on the Security Council), they do earn more blame than, say, Zimbabwe.
Anyway, the app was presumably made for a government contract, and silliness like this is common with those. There's a long list of contractual requirements for things made for the government, and the vendor has to meet all those requirements, so that's what they do. The vendor only has to satisfy the demands of the customer, and some 101-year-old person is not their customer, nor is whatever agent who had to deal with the mess, the government is (specifically whomever awarded the contract).
Generally speaking, with software, it works best when the customer is also the user. Users who don't like some software will try to avoid buying it, or will find another vendor if they're mad. When the customer is someone totally different from the user, you see real problems in usability, because the customer doesn't use it and doesn't care. So software made for government contracts tends to be horrible.
> and anything with "20" or greater was science fiction and would never happen in our lifetimes
Or rather, "this stuff won't be used in 2000 or later, until then we will have fixed it"... well, many banks, airlines and other megacorps still run on Cobol code which is decades old today.
That’s clever, but still leaves a significant edge case. I’d hazard to guess that 101-year-olds don’t travel very much and are therefore rather likely to be forced to get new passports for this specific purpose. (But perhaps the home office accepts expired passports?)
>But I doubt this 101-year-old got much choice in exactly what format his passport contained his DOB
I’m sure he didn’t, this definitely isn’t his fault.
> so the software needs to handle it
Does it though? It’s a rare edge case that should be trivial to solve with minor human intervention.
> Does it though? It’s a rare edge case that should be trivial to solve with minor human intervention.
Apparently[1] centenarians make up 0.02% of the UK population; how well this statistic transfers to EU nationals in the UK I don't know, but if it's 1:1 then this issue affects about 600 people. A large proportion of those will however have some trouble rectifying the problem themselves; the over-90-year-olds I know certainly would all struggle with this sort of thing. In the article, the activist who volunteers helping people with the registration process is quoted as saying it required 2 calls to the Home Office. This is someone who presumably has some practice dealing with the process and the Home Office and they couldn't get it sorted out trivially. If it's an accepted failure case of the automated process, there needs to be an established alternative process; it sounds like no consideration was given to this edge case at all. So, not "minor human intervention."
If by most, you mean >50%, then you're probably right, but but especially within EU countries, having >10% foreign nationals among the population is common. Those foreign nationals will frequently give birth to equally foreign children, who need to obtain passports whether or not they're planning to travel. (Although they frequently do travel to visit family.)
Title text: ISO 8601 was published on 06/05/88 and most recently amended on 12/01/04.
"The title text provides a perfect example of the kind of ambiguity that can arise when non-standard formats are used. The ISO standard was in fact published on 1988-06-05 and amended on 2004-12-01. This is mentioned in the title text in MM/DD/YY format; however, there is no way to naturally figure this out, particularly with the second date. With the year truncated to two digits and all three numbers at 12 or lower, the date referring to December 1, 2004 may well be interpreted as 12 January 2004, or as 2012-01-04."
The Home Office is a den of mad, evil bureaucracy. I'd honestly have believed it even if it hadn't been anything to do with a computer. The people who work there are especially twisted, the culture needs flushing down a toilet.
I worked on a system that processed benefits. We generated a lot of reports, one of which being the overage dependent report, listing all children that were over the age limit for that insurance plan.
We had one case where a newborn was showing up on that report, and for end users, there was no indication why. DOB was 20-something days ago, not 20-something years ago.
Lo and behold, the paperwork was somehow filed before the baby was born, and our date calculation method was returning an unsigned int. So the logic was resulting in an age of -1 (since DOB was prior coverage start date), which wrapped around to 255, which was obviously higher than whatever the max dependent age was.
It was a fun one to debug and explain.