If you're confused about how the hell AI is going to make the kind of money needed to turn a profit, I'll illustrate

[Metaphorical]

2. I have worked with various forms of AI since the 1980s.

There are several major flaws in your post. The first is that LLMs in general have an accuracy rate of approximately 70% (across the board, OpenAI is less accurate than others). This means 30% of the time, the information that you receive from an AI will be wrong in some critical way. There are many sound mathematical reasons why this is the case, and it's pretty fundamental to the underlying transformer model. This has been known for a decade or so. AI can be useful - I use it myself for intellisense, when I know I can reasonably count of the underlying patterns, but even here, the benefits that I get back from such LLMs need to be weighed in terms of how much additional time I am now spending analysing the results to make sure that what I'm getting back is valid, and correcting it when it's now.

OpenAI does not "self-train". It (and others like it) typically employ many people (at very low wages) to filter out and "pretrain" their data, often at considerable psychological distress; that work means that much of the hard task of classification has already been done, but it's something that is in fact not sustainable. There have been many attempts to generate content that can be used for pre-training; however, because of the nature of the way that latent spaces generate narrative threads (something I won't get into here), what happens is that the mock training data usually loses a lot of intrinsic context, becoming blander and smoothed down over time, much like multiple repetitions of copying the output from one copying machine by the same mechanism.

Finally, we have effectively used the bulk of the Internet to train the models used for things like ChatGPT 4 (yes, we're up to 5, but the document corpuses have not changed significantly) and this means that we're seeing a plateuing of improvements in design in particular.

There are some interesting areas of research (especially into world models) that I suspect may provide a better approach to GenAI, but right now the consoritium of the Mighty Seven that effectively support AI are reluctant to go down that road because it is not beneficial to their longer term goals of getting the American public to build data centres for them.

Yes, AI might (almost certainly will) improve, but its probably not with this architecture, nor with the incredible amount of very questionable financial plays going along behind the architecture.