> Ignorance of your "black box" excuses not

Hopefully the FTC understands linear separability. Because I often get the impression that people who want ML models to be explicable don’t, and are expecting the mathematically impossible.

I rather think that they (the regulators) are saying that you can’t expect to use mathematical impossibility as a legal argument to avoid liability if your AI does material harm.

Doesn't this kill AI as a product?

Or at least mean that if you're too poor for a doctor or lawyer you can't fall back to the unreliable AI?

Not if you "keep your claims in check" and have proper oversight of the output your AI generates. Then you will be ready at all times to step in, should something go south. Yes, maybe it does not scale up unlimited with proper oversight in place, but this is exactly what Google and co are missing these days and the reason they extract cash to no end. Proper oversight is completely missing.

Aren't they really only saying you have to disclose it in your claims about the product?

Exactly my read - they're just saying that if you want to pretend you are selling magic that's not allowed.

Quantifying your risk does not mean you eliminated all your risk.

We allow the sale of alcohol, tobacco, and firearms too.

that is correct.

If the system your building is "mathematically impossible" to explain in a way that would be expected of every other system. Maybe you shouldn't be building that system.

Mathematical explanation is not required in many areas.

Perhaps you can’t perfectly explain it but you can at least understand it statistically.

For example is not currently possible to mathematically explain people’s behaviour, but there is statistical evidence and also accountability on an individual. Eg a doctor making a decision about a scan result.

It's even simpler than that. You don't need any particular level of understanding, you just can't lie about what your level of understanding is.

The following would seem to be OK: "This model performs well in a set of tests we devised, your mileage may vary."

I raise another example, to question, you can statistically solve the famous diffusion problem.

Non-linear models often perform better. You’re never going to approach anything like GPT with a linear model.

I don't think mentioning non-linearality means you can bypass the singularities you may cause typically CUSP, buckling etc and omit the analysis intentionally or negeliccantly.

Theoretically you could use a polynomial model which, while not linear, is pretty high on the explainability scale.

From the FTC's perspective, stuff like that is just a technical detail. It is on the business to make sure that the final result can be explained.

If you can't explain the ML part due to whatever math detail, you better make sure you write a good wrapper around it to catch bad output which you can explain.

Lots of businesses deal with high complexity and open ended problems that lead to "inexplicable" risks. They address this by showing diligence in monitoring and mitigating risks, evolving their approach as new concerns are understood. AI companies should employ similar motions (and some do).

Actually, if you ask GPT3 to explain step by step how it arrived to the conclusion, it works pretty well and even has higher accuracy.

GPT is making it up.

It’s cobbling together an explanation from texts that explain how to solve a problem. It’s like a student who copies an answer, then copies the explanation as well.

Chain-of-thought is not the same with direct prompting. Step by step is better.

> Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking.

https://arxiv.org/abs/2201.11903

But there’s no reason to believe that is actually how the result was generated.

The result was generated via a deep neural network, not by whatever explanation or reasoning it prints out when asked.

As someone who don't understand linear separability (at least in the way you're using it), can you explain?

Why bother quoting when it's the entire comment? Never understood that