Proposed amendment to legal presumption about the reliability of computers

Houses of Parliament, taken on 26 Nov 2024

I am grateful to the journalist Tom Webb, who specialises in data protection, for alerting me to an amendment to the Data (Use and Access) Bill, currently going through the House of Lords.

It concerns the legal presumption that “mechanical instruments” (which seems to be taken to include computer networks) are working properly if they look to the user like they’re working properly. This has come in for quite a kicking in recent years. I was first alerted to it in 2013 by the barrister Stephen Mason.

Mason has spent longer than a decade telling anyone who will listen it is a deeply flawed legal presumption. Here he sets out his reasoning. Mason has won some influential supporters along the way, including Lord (James) Arbuthnot who has also spent longer than a decade campaigning on behalf of Subpostmasters affected by the Post Office scandal.

Although the “mechanical instruments” presumption has never, to the best of my knowledge, been quoted in any civil or criminal proceedings involving a Subpostmaster, it has been said to effectively reverse the burden of proof on anyone who might be convicted using digital evidence.

The logic being if the courts are going to assume a computer was working fine at the time an offence allegedly occurred because it looked like it was working fine, it is then down to the defendant to prove that it was not working fine. This can be extremely difficult to do (per the Seema Misra/Lee Castleton cases).

This is the amendment proposed by Lord Arbuthnot and others. I am grateful to Tom for bringing it to my attention. It says:

After Clause 132, insert the following new Clause –

Reliability of computer-based evidence

(1) Electronic evidence produced by or derived from a computer, device or computer system (separately or together “system”) is admissible as evidence in any proceedings –

(a) where that electronic evidence and the reliability of the system that produced it or from which it is derived are not challenged;
(b) where the court is satisfied that the reliability of the system cannot reasonably be challenged;
(c) where the court is satisfied that the electronic evidence is derived from a reliable system.

James Arbuthnot

(2) Rules of Court must provide that electronic evidence sought to be relied upon by a party in any proceedings may be challenged by another party as to its admissibility.

(3) For the purposes of subsection (1)(b), Rules of Court must provide for the circumstances in which the Court may be satisfied that the admissibility of electronic evidence cannot reasonably be challenged.

(4) When determining whether a system is reliable for the purposes of subsection (1)(c) the matters that may be taken into account include –

(a) any instructions or rules of the system that apply to its operation;
(b) any measures taken to secure the integrity of data held on the system;
(c) any measures taken to prevent unauthorised access to and use of the system;
(d) the security of the hardware and software used by the system;
(e) any measures taken to monitor and assess the reliability of the system by the system controller or operator including steps taken to fix errors or address unexpected outcomes including the regularity of and extent of any audit of the system by an independent body;
(f) any assessment of the reliability of the system made by a body with supervisory or regulatory functions;
(g) the provisions of any scheme or industry standard that apply in relation
to the system.

(5) For the purposes of this section –

computer” means any device capable of performing mathematical or logical instructions;
device” means any apparatus or tool operating alone or connected to other apparatus or tools, that processes information or data in electronic form;
electronic evidence” means evidence derived from data contained in or produced by any device the functioning of which depends on a software program or from data stored on a computer, device or computer system or communicated over a networked computer system.”

Member’s explanatory statement
This amendment overturns the current legal assumption that evidence from computers is always reliable which has contributed to miscarriages of justice including the Horizon Scandal. It enables courts to ask questions of those submitting computer evidence about its reliability.

To read the amendment within the list of Lords amendments to the Bill so far, click here and scroll down to page 45.


The journalism on this blog is crowdfunded. If you would like to join the “secret email” newsletter, please consider making a one-off donation. The money is used to keep the contents of this website free. You will receive occasional, irregular but informative email updates about the Post Office Horizon IT scandal.

13 responses to “Proposed amendment to legal presumption about the reliability of computers”

  1. Just to add a bit of flavour to the discussion about Reliability/Unreliability of computer-derived evidence and ‘robustness’ (whatever that means), perhaps I could relate a true story about a software bug which only showed itself every four years, and then only in specific circumstances. The owners of a large mainframe-based system ordained that their ‘customers’ should be permitted to continue to use application forms intended for computer processing containing dates in DD/MM/YY format, just avoid imposing too much change to local procedures. The mainframe system, however, had to store dates as a number of days from 1 January 1900 in a hexadecimal format, so the ‘front end’ of the system employed bespoke software to convert YY to YYYY, which worked well but not if the processing took place on a working day (mon-fri) which happened to be 29 February, in which case the bug caused dates to be truncated by 100 years. The bespoke software was written in the early 80s and we did not identify the bug properly until 1 March 1996. I suspect that in fact the YY/YYYY conversion was copied over from the previous version of the system which was written in the late-60s, so the ‘Leap Year Bug’ (possibly) remained hidden for the better part of 30 years.

    The results of the bug were found in 1996, because by then we were exporting dates to another mainframe system which quickly identified and reported the errors.

    1. Amazing – thanks Ken. That’s so clear. I shall use this as an example next time someone suggests that computers are like calculators and 2+2 always = 4.

  2. HL Bill 40: Comment on proposed clause “Reliability of computer-based evidence”

    This initiative is most welcome, I have been living with these issues as an academic and expert witness since the early 1990s. But a simple reversion to s 69 PACE won’t work because there were too many exceptions and arguments about who was qualified to provide a computer with a certificate of normal working – in one famous example the certifier was a Marks & Spencer store detective.

    There are some practical weaknesses in the current proposals::
    1 The ability of a court to be satisfied under subsection (1) or for challenges to be mounted under subsections ( 2) and (3) depends on the ease with which disclosure is available to a challenger. Theoretically disclosure is fully supported by Civil Procedure Rule 31 and Criminal Procedure Rule 15 but in both instances there are ample opportunities for those expected to disclose to slow down the process and build up the costs of those seeking disclosure. Grounds for resisting disclosure can include that what is being sought is irrelevant to the issues, confidentiality and disproportionate cost. Disclosure may take place in stages. Lawyers will need to be able to fund experts to see that disclosure requests are based on careful research and are properly framed; they may need the further support of those experts where a judge is asked to adjudicate. In criminal cases under legal aid such experts may be difficult to source; in civil cases there is often little parity of resource between Parties.
    2 It seems unfortunate that the Procedure Rules largely assume that expertise will be provided by the Parties whereas, given the complexity of many computer systems and the many stages that might be involved in generating “documents” to be provided as evidence, it is judges who need direct support. In my experience the abilities of judges in these circumstances is variable.

    Quite rightly the main test case of computer admissibility and reliability issues is Post Office/Horizon but policy makers ought to look at Operation Venetic/EncroChat. This was/is the largest ever UK law enforcement operation with over 700 arrests. Evidence was produced from encrypted smart phones by a Dutch/French team but the French refuse disclosure of the precise methods which involved remote hacking – equipment interference – and subsequent processing prior to delivery to the National Crime Agency. In these cases the “victims” do not immediately attract public sympathy – they have been accused of importation and wholesaling of large quantities of cocaine; in some instances there have been accusations of conspiracy to murder. The French have asserted the reliability of their methods yet computer-aided examination of the tendered output shows many instances of failure and inconsistencies. Prosecution experts have agreed and accepted that these failures occur. Judges, up to the Court of Appeal, have decided that the EncroChat evidence is admissible. But the trials of accuseds have gone ahead and with many convictions.

  3. I programmed online banking, telephony and databases among other endeavors during my IT career.

    Databases generally have a high degree of integrity built in. Networks much less so, especially when you are using land lines in rural areas. My own town of 100,000 has frequent outages and line hiccups. Telephony has its challenges.

    A proper audit requires network and transaction logs at both ends. The first task is finding dropped and inadvertently duplicated transactions.

    The banks and payment processors generally do a sterling job of getting transactions done right. But mistakes in large organizations do happen. One of my performance modifications tested well in a single thread environment. It was put into production without my advance knowledge. My supervisor and I kept our fingers crossed, but the clerk responsible for checking end of day transaction counts didn’t. Some thousand dropped transactions later, we caught on.

    Every few weeks the mainframe went into an interrupt loop shutting down the entire online banking system. It took me months of wading through thousands of listing pages to find where a register value could very occasionally get improperly reset as a result of poor design by IBM developers.

    Errare programmer est.

  4. Absolutely. A total figure is not evidence: a transaction log creating that figure is.

  5. More and more, I’ve come to conclude that the combined incompetence/fraudulence (as appropriate) of POL and its contractors was only the second point of failure and that the primary failing was the way in which the courts handled the defective cases. It has been noted before that the reversal of the PACE 1984 requirements for computer-based evidence was enacted just as the Horizon system was being created, and with the PO side rooting for the change (the posts at davidallengreen.com linked by Hazel above are one example).

    Good practice in software development, sufficient to give confidence, or otherwise, in the resulting product, significantly pre-dates 1999, and would doubtless have been in the minds of those guiding the 1984 requirements. In fact, today’s software development (think: almost every website/mobile application that you use) may often be less capable of being relied on for data of evidential quality than that of the PACE s.69 era.

    The proposed amendment is obviously an improvement, but:
    * clause 1(b) is an obvious loop-hole that should be removed, unless the Rules under clause 3 would always allow challenge;
    * clause 4 is phrased as guidance that has no legislative force (but IANAL) and could be removed, or it should say “… the matters to be taken into account …”
    * no similar clause/Note is included to illustrate what could lead to clause 1(b) being satisfied.
    * should there not be a requirement for an independent assessment of any system whose correct operation is necessary for evidence being adduced, where (1) costs of assessment are to be paid by the party claiming correctness and (2) the claim of correctness is denied if inadequate evidence is made available to the assessor?

    Or why not just restore (ie repeal s.60 of the1999 YJCE Act and reinstate) PACE(1984) s.69, for which there is already 15 years of precedent?

    1. I disagree: ground zero is the unfair contract which makes the SPM guilty until proved innocent.

  6. I assume the intended meaning of sub-clause (1) is as though the words “and not otherwise” were to appear at the end of the sub-clause – ie the effect of it is to limit rather than enlarge the class of admissible evidence.

    And what does sub-clause (2) mean – I mean, isn’t what it says always true whether such a provision appears in the legislation or not?

  7. I thought legislation like this should read either
    (a) … ;
    (b) … ; or
    (c)

    or

    (a) … ;
    (b) … ; and
    (c)

    If I’m wrong, which is the actual meaning of the proposed legislation?

    1. Agreed. Mere semi-colons do not suffice to convey the intended meaning.

  8. David Allen Green has also written cogently about this, and the difference it might have made to the initial legal outcomes for sub-postmasters if the presumption re computer reliability had been the other way during the years of prosecutions: https://davidallengreen.com/2023/09/computer-says-guilty-an-introduction-to-the-evidential-presumption-that-computers-are-operating-correctly/

  9. Initiatives to overhaul the legal presumption about the reliability of computers are essential and long overdue. After almost 50 years’ experience working in industry on large and complex system employing computers I am in no doubt that there will always be both residual bugs, errors, & defects as of their initial deployment, and latent defects which appear on a long-tail distribution. Latent defects often appear years later when the original system is flexed to meet new operational requirements or is updated to deal with obsolescence. This problem has be proactively managed through appropriate organisational governance, policies, process, training. Post Office Limited never did understand this, nor did the listen to advice.

  10. One omission is an onus to reveal what automated test cases have been used to determine system behaviour. This would give expert witnesses the information they need to at least understand what the manufacturer was basing claims on.

    I’d like to see a right to see test suites, transaction and observability logs. I might go as far as assuming the system did not work as described if these things were lacking.

    This has struck me the whole way through. A 1999 system predates modern development techniques, yet somehow, the supplier got away with merely claiming it was fine. These days, there should be evidence during development and operation that could back up such claims. Or not, in the case of Horizon – it is clear much of this simply was not done.

Leave a Reply

Your email address will not be published. Required fields are marked *

Archives

  • 2024
  • 2023
  • 2022
  • 2021


Subscribe For Latest Blog Updates

Tags

Alan Bates alice perkins Alwen Lyons Andrew Winn Andy Dunks Andy Parsons angela van den bogerd Bates v Post Office BBC Bonusgate CCRC Chris Aujard Clarke Advice False Accounts Fujitsu Gareth Jenkins Grabiner HCAB Horizon Hugh Flemington Inquiry Interim Report Janet Skinner Jarnail Singh Kevin Hollinrake Lee Castleton Lord Arbuthnot Mark Davies Nicki Arch Nick Read Noel Thomas Paula Vennells Paul Marshall Post Office Rob Wilson Rod Ismay Rodric Williams Second Sight Seema Misra ShEx Simon Clarke Susan Crichton Tom Cooper Tracy Felstead UKGI

Categories