Fraud-Fu and the Great Algorithmic Toilet Bowl

Audio article

Checking narration… Playback speed 0.75×1×1.25×1.5×1.75×2×

Checking article narration…

Apparently artificial intelligence is going to make criminal hacking a career for everybody. Sit in a rented room, ask the machine to do the clever bits, steal enough money to escape the rented room, retire somewhere inconvenient for the police. Somewhere in this magnificent plan, the aspiring criminal has mistaken access to a tool for possession of a functioning business, which is a mistake people make in perfectly legal businesses too, although the consequences usually involve fewer searches of the rectum.

Before we all enroll in advanced fraud-fu, I should remind you that AI was protecting payment cards long before OpenAI gave the public something to argue with. FICO’s Falcon fraud-detection system was introduced in 1992 using neural networks. Nineteen ninety-two. The machine was already looking for suspicious transactions while a great many people were still trying to understand why anybody would want a computer in the house.

It just wasn’t writing poems about itself.

The work was comparatively unglamorous. Look at a transaction, compare it with patterns in the available data, estimate how suspicious it is. An unusual amount, a sudden burst of purchases, a combination of circumstances that deserves another look. Alongside other checks, that score can help a bank decide whether to approve a payment, challenge it, or stop it. There are people involved, rules involved, mistakes involved. Sometimes the suspicious bastard is you, legitimately trying to buy something, and now you have to persuade your bank that your own money should be allowed to accompany you outside.

But this machinery existed. It was doing useful work before AI became a face on a screen, before everybody started asking whether it had feelings, before we decided that a machine explaining a sandwich was a more interesting achievement than a machine helping prevent somebody from emptying an account.

Nobody writes poems about the plumbing. The humble commode carries away your daily shame without demanding applause, and you remember its contribution to civilization only when it stops working.

Unfortunately, the shit can use AI too.

That part is real, and pretending otherwise would be stupid. Generative AI can help criminals write convincing messages, translate them, impersonate people, produce fraudulent images and voices, and do some technical work with less effort. The badly written scam email is losing its usefulness as a warning sign. A thief can now sound educated, patient, sympathetic, all the things that ought to make us feel better about a conversation and increasingly do not.

So yes, some barriers are coming down. You do not need a roomful of expensive equipment to deceive somebody. You do not need to invent every tool yourself. Criminal services existed before chatbots, and buying or renting somebody else’s work is hardly an innovation in either crime or capitalism.

What does not follow is that everybody with a chatbot has been handed a reliable income, an escape from consequences, and a little villa by the sea.

There is a considerable amount of life between getting a tool to produce something and getting the result you imagined. At the more demanding end of cybercrime, there is still technical understanding, testing, coordination, maintenance, and the recurring discovery that something you depended on no longer works. A tool can save labor without removing the need to understand what it is doing. Outsourcing the clever bits also leaves you dependent on whoever sold them to you, and you have chosen a market in which dishonesty is not exactly a regrettable deviation from professional standards.

The man selling you the shortcut can be a fraudster too. I feel this possibility deserves a little more attention in the business plan.

And the equipment is only part of the expense. There is the time spent learning, the time spent failing, the people taking their cut, the money you may lose to your own associates, the effort of keeping an operation going while somebody else is trying to stop it. The costs vary enormously. A crude scam and a sophisticated intrusion are not the same undertaking, and putting them under the same heading does not make their economics identical.

I cannot give you an honest calculation proving that every criminal loses money. Some criminals make enormous sums. Organized ransomware groups have collected millions without being national intelligence agencies. It would be comforting to announce that crime has become financially unviable, but the people whose money disappeared might reasonably ask where the fuck it went.

What I am objecting to is the cheerful leap from “this has become easier to attempt” to “this has become a sensible way to make a living.” Those are different claims. A headline about a stolen fortune tells you very little about your own chances of acquiring one, keeping it, or remaining free to enjoy it.

You thought you were buying a career. You may have bought a treadmill wired to a woodchipper, and the fellow selling replacement blades would very much like you to keep running.

Meanwhile, the people protecting the money have computers too.

This seems to get forgotten in the excitement. The bank has not been sitting there with a padlock and a retired watchman, waiting for your chatbot to invent deception. Machine learning can help identify suspicious transactions and activity across more data than a person could inspect manually. Newer AI tools can help security teams examine code, investigate alerts, and make sense of the enormous quantity of information that otherwise lands on somebody’s desk and quietly becomes next week’s problem.

Used well, these tools can make particular attacks harder, expose patterns, and shorten the time available to exploit a weakness. Your clever little trick may have a shorter useful life than you expected.

But the entire banking system does not become an immortal hydra because somebody bought an AI product. Models miss things. They flag innocent people. Organizations run old systems, delay repairs, fail to connect information, and occasionally buy something impressive without doing the tedious work necessary to make it useful. An algorithm cannot approve its own budget or persuade every department to stop protecting its little patch of institutional stupidity.

Nor does every defensive system instantly learn everything every other system has discovered. Information has to be collected, checked, shared where possible, and acted on. Somewhere in this apparently frictionless future, a human being still has to decide whether shutting something down will stop a thief or stop the hospital.

The attacker is not obliged to choose the best-defended target, either. There are weaker systems, neglected systems, and people who can be persuaded to authorize a payment themselves. A bank can correctly establish that it is you pressing the button while failing to establish that somebody has spent the last hour terrifying you into pressing it.

That is a particularly foul part of the business. The transaction can be technically authorized and the person making it can still be getting robbed.

The criminal does not always need to defeat the machine. Sometimes he needs to convince a frightened person that their son is in trouble, or a lonely person that somebody finally gives a damn. AI can help dress the lie. The victim supplies the fear or hope because they are human, and there is nothing ridiculous about either until some bastard learns how to invoice it.

Then there are the protected operations, the examples people produce as if they have discovered a loophole in the laws of personal consequence.

North Korean state-backed cryptocurrency theft is real. But a state conducting theft with organized personnel and protection is a very different proposition from you buying software and hoping to become independently wealthy. You cannot add government protection to your circumstances merely because you have read about somebody else possessing it.

And you do not need a state to build a serious criminal organization. That claim would be just as wrong. Money, specialization, corrupt protection, and networks of people doing different parts of the work can sustain operations well beyond the capabilities of the solitary genius in his rented room. The fantasy keeps showing you one man with a laptop. The actual machinery can involve a great many people, and your place in it may be considerably less glorious than the advertisement suggested.

Look at the scam compounds in parts of Southeast Asia. Some of the people behind those screens were lured by job offers, trafficked, confined, and beaten into defrauding strangers. Calling all of them enthusiastic career criminals is a convenient way of overlooking the people who own the compounds and collect the money.

A person being tortured for missing a scam quota has not discovered the liberating possibilities of remote work.

There is your digital economy with the skin peeled off: somebody desperate for a job is imprisoned and forced to prey on somebody desperate for affection, or frightened of losing their savings, while the people arranging the whole obscenity take their share. Better software can be fitted into that machinery without making any of it less obscene.

And no, I cannot promise that the police arrive at the end of every story. Investigations can succeed. Infrastructure can be seized, associates arrested, records recovered. Other offenders remain out of reach. Jurisdiction, corruption, resources, and protection matter. Justice does not descend automatically because an algorithm has become sufficiently annoyed.

But uncertainty about being caught is still uncertainty. It does not become safety because the money arrived once. Your associates can be investigated. Your protection can change. Something you thought had disappeared can turn up later. A successful theft does not come with a certificate declaring the rest of your life settled.

So yes, the Tom-and-Jerry business will continue. Tom, the repeat offender, comes back bandaged and bitter with another elaborate device, and Jerry has had time to rearrange the furniture. There will be traps, explosions, expensive ingenuity, and the recurring humiliation of discovering that the other fellow has also been thinking.

Except Tom has AI now, and so does Jerry. Sometimes Tom wins. Sometimes Jerry mistakes the homeowner for Tom and locks him out of his own kitchen. Sometimes nobody has patched the bloody cat flap.

The cartoon cannot settle the argument for us. There is no animator obliged to make the defender win, and no reason to assume that faster tools automatically produce a safer world. They can improve protection. They can improve the attack. What happens depends on who uses them, what they can see, what they are allowed to do, and whether anybody acts before the money is gone.

The invisible machinery has been helping protect us for decades. It will keep improving. So will some of the machinery trying to rob us. Neither development owes the aspiring criminal a comfortable livelihood, and neither gives the rest of us permission to stop paying attention.

You are still the warm-blooded wallet that walks and talks and buys things. The bank has reasons to protect that wallet. The criminal has reasons to empty it. Both can now bring increasingly capable software to the argument.

And somewhere, while everybody admires the technology, somebody is staring at an account that no longer contains the rent. The caller sounded convincing. They thought they were doing the right thing. Now they are replaying the conversation, trying to find the moment when they should have understood, as if finding it will put the money back.

They do not give a shit which side had the better model.