[hrothgar]

Nic Hammond was gracious enough to sell me one of the early copies of Detecting Cheating in Bridge. The following is not intended as a comprehensive review or evaluation of his book; rather, I am going to provide a few observations regarding portions of the work that I found particularly interesting.

TLDR: Provocative read. Glad I purchased / read this. It will be very interesting to see what comes from this.

High Level Summary: Nic makes a number of claims including

Techniques that rely on “breaking the code” to infer that cheating is taking place are insufficient in the face of an adaptive adversary who does not cheat in a consistent / reproducible manner.
However, it is possible to use statistical methods to infer that cheating is taking place and that specific pairs are cheating.
These methods focus on whether players are too successful at making what should be “random” guesses.
He has an expert system that is instantiates a number of detection routines. This expert system successfully flags well known pairs that people believe to have been cheating in the recent past.
The expert system strongly suggests that not all of the pairs who were cheating prior to 2015 have been identified.
The expert system strongly suggests that the amount of cheating decreased post 2015, however, it has not been eliminated.


What is / is not included in the book:

Most of the data is anonymized. Nic provides data regarding the performance of pairs such as F/S and F/N and uses these as reference points. However, for the most part the presentation focuses on the shape of various distributions rather than the identity of specific individuals.
Specific details of the algorithms that the expert system uses to infer that cheating is taking place are not provided
My (strongly biased) thoughts:

Anyone reading this book will need decide whether or not they are willing to make at least two major leaps of faith:

Do they believe that the statistical tests of the sort that Hammond proposes are sufficient to infer cheating?
Are people confident in Hammond implementation of these tests


For me, the answers to these two questions are both yes (however, I’m going to caveat this almost immediately). With respect to the question about statistical validity, I’d like to focus on a couple different aspects of Nic’s analysis.

First: I very much like the fact that Nic was able to create a specific hypothesis that he could examine. To wit; the fact that we can divide the data set into observations from before 2015 and observations after 2015. (2015 being the year in which Boye went public with his cheating accusations and the start of much wider video surveillance) Given that the risk / reward ratio for cheating has changed significantly, it seems reasonable to test whether the frequency of cheating has decreased.

Next: If I understand things correctly, it should now be possible to inspect the hands that WILL be used for individual tournaments, identify boards that the expert system will use, and then generate a hypothesis in advance regarding pairs that are unusually likely to make the right decision.

I have no doubt that other people will question Hammond's claims. (I admit to having some questions myself, but want to wait until I am able to able to think about stuff more thoroughly) However, my preliminary assessment is as follows:

1. I do not believe that the methods that Hammond proposes are sufficient to prosecute individuals or pairs for cheating, strip titles, or make public accusations. I don’t see this as an indictment of the methods themselves, but rather, if folks are smart about their choice of the set of hands on which they chose to cheat it will be almost impossible to prove these sorts of claims definitively. (Hammond himself notes that smart cheaters will be cognizant of these sorts of trade offs) In the end, Hammond’s system may be more useful as a deterrent than as a detection system

2. I am not particularly interested in demonstrating with absolute certainty that pair foo cheats. To me, the real value of this study is to try to establish a rough consensus regarding whether or not the game as a whole is “clean” or “dirty”. Hammond’s work suggests that cheating in top level events was pervasive and, even today, the amount is problematic. (Note that Hammond's methods can be applied to entire populations in addition to exemplars)

3. To me, it seems clear that we can not trust participants not to cheat. We can not rely on “code breaking” to ensure convictions or even act as a sufficient deterrent. Its unclear whether Hammond’s methods will stand the test of time.

4. I’ve said it before, I’ll say it again: If folks are serious about rooting this out, it requires switching to an electronic playing environment.

[nige1]

Thank you, Hrothgar, for posting this.
I agree that the long-term solution is (4) but in the mean time video-surveillance and work like Hammond's is valuable.
Also, In the past, on BBO etc, I've made some suggestions...

• Automatically monitor carefully selected hands. Authorities could further investigate players whose actions, on those hands, were suspicious.
• Automatically build up a "convention card" for regular pairs to see if their bidding/play explanations tallied with what they divulged.
• Designate an official and give him the authority and means to investigate cheats. The WBF should be responsible for investigating top-level cheating because NBO's protect their own international cheats. For example, peruse Bobby Wolff's posts about how the ACBL handles domestic cheats in this BridgeWinners article,
• Define an adequate legal protocol to investigate and prosecute cheats. Make sure that NBO/WBF constitutions/COCs specify that members/competitors agree to necessary investigation/prosecution/sanction methods, Ensure that prosecutions are conducted in a fair, timely, efficient, clear, robust way, so that their conclusions can withstand review by independent arbiters. In the 1965 Reese-Schapiro case, the WBF procedure was fundamentally flawed, in many ways (e.g. the chairman of the appeal committee was a prosecution witness). Over the past 50+ years, there has been scant improvement. Recent efforts, by the WBF, EBL and NBOs have been farcical (Israel an honorable exception). Urgent official action is needed.

[cherdano]

 hrothgar, on 2019-July-19, 10:45, said:

4. I’ve said it before, I’ll say it again: If folks are serious about rooting this out, it requires switching to an electronic playing environment.

Wow, sounds like the book *really* changed your mind!

[hrothgar]

 cherdano, on 2019-July-19, 17:23, said:

Wow, sounds like the book *really* changed your mind!

Hey, I did say my opinion was biased....

[cherdano]

On a more serious note, can you say just a little bit about what he is doing? The excerpt that he published makes it seem as if his methodology is "If your defense is better than Lauria-Versace, then you are cheating!" (Where "better" is probably measured by the number of tricks dropped compared to double dummy defense per session.)
While this may well be empirically true, this method could NEVER work as a method of convicting cheaters - you just cannot have a rule "You are not allowed to play defense better than Lauria-Versace".

[nige1]

 cherdano, on 2019-July-19, 17:35, said:

On a more serious note, can you say just a little bit about what he is doing? The excerpt that he published makes it seem as if his methodology is "If your defense is better than Lauria-Versace, then you are cheating!" (Where "better" is probably measured by the number of tricks dropped compared to double dummy defense per session.)
While this may well be empirically true, this method could NEVER work as a method of convicting cheaters - you just cannot have a rule "You are not allowed to play defense better than Lauria-Versace".

I haven't read Hammond's book but I understand that

• He studies unusually good performance to create a short-list of possible cheats but then
• He searches the short-list for successful, against-the-odds, actions on selected deals, to distinguish probable cheats.

I suppose the corollary is that pairs who perform much worse than usual/average aren't cheating. Hence, this methodology might have exonerated Reese-Schapiro

See Hammond's comments in this BridgeWinners thread for example ,,,

Nicolas Hammond said:

... Suppose setting the contract relies on switching to a certain suit in the middle of a hand. Dummy/I have the same cards in both suits. It is a 50-50 guess. A cheating pair will get this right more than 50% of the time. One hand says nothing; multiple do ...

[steve2005]

While this program may cause you to believe a pair is cheating there is no proof.
The program, can only be a guide on who to look at further with video surveillance and other methods.

[pescetom]

 steve2005, on 2019-July-19, 21:09, said:

While this program may cause you to believe a pair is cheating there is no proof.
The program, can only be a guide on who to look at further with video surveillance and other methods.

You don't necessarily have to accuse them of cheating, you could suspend them due to anomalous luck patterns - similar to when cyclists were suspended for hematocrit exceeding 50%. It would be similarly unsatisfactory, but better than nothing.

[hrothgar]

 steve2005, on 2019-July-19, 21:09, said:

While this program may cause you to believe a pair is cheating there is no proof.

So what? I'm not interested in proof or even prosecution.
I am interested in seeing folks implement changes that make it harder to cheat and this requires a very different standard.

Quote

The program, can only be a guide on who to look at further with video surveillance and other methods.

Clear cut "obvious" video was insufficient to convince the CAS regarding the set of methods folks were using to cheat three years back.

Once the adversary switches to more complex codes / decreases the number of hands where the cheat this video will be useless.

[hrothgar]

 cherdano, on 2019-July-19, 17:35, said:

On a more serious note, can you say just a little bit about what he is doing? The excerpt that he published makes it seem as if his methodology is "If your defense is better than Lauria-Versace, then you are cheating!" (Where "better" is probably measured by the number of tricks dropped compared to double dummy defense per session.)
While this may well be empirically true, this method could NEVER work as a method of convicting cheaters - you just cannot have a rule "You are not allowed to play defense better than Lauria-Versace".

The algorithms being used are really opaque.

The more interesting stuff is related to the charts, especially the ones showing distributions of outcomes and the degree of separation between various pairs.

[thepossum]

The problem I have with many of the current approaches to algorithmic data science for issues as important as detecting cheating are that they need to be open, explainable and not highly prone to error. The level of predictive capacity of many statistical methods would need very careful use to avoid risk of tarnishing people's reputations, or alternatively tarnishing the reputation of those who use them. Combination of analytics and expert systems maybe provides some hope, since expert system approaches have been successfully used in legal and explanation and argument applications. There is a rather alarming tendency in the world for very import decisions to be taken out of hands of experts, for people who do not understand statistical limitations to overuse things, and for potential massive injustice. I'm particularly concerned about this is anything involving law, policy, access to benefits etc., but alleged cheating bridge is clearly a place where justice is essential. We risk, in this world, losing understanding, explanation, due process etc. under the recent trend towards predictive capacity

I will certainly look out for the book. I have been trying to find any quality published statistical theory on bridge analysis - that is beyond the basic stuff we usually all think of as bridge statistics.

I would be interested in any pointers to quality theory and writing on the very complex issue of competitive bridge. It strikes me, after less than 12 months thinking about it, that the nature of the game, tournaments, the sources of variance/error, the data/information requirements are massive.

Will follow this with interest. I am rather concerned by a few things you describe hrothgar. Many algorithmic approaches are seriously flawed if the approach to training is flawed and end up with entrenched biases and injustices - especially using other people (possiblly competitors) beliefs about cheating.

From my knowledge, so far, I have not seen the theory clearly enough expounded to even start analysing it in this way. Also, understanding how much data and analysis would be required to prove in a alegal or quasi legal sense

Very interested in your future posts