No, unfortunately most of my work is with tabular data with a bit of computer vision- I haven’t looked into any application of language models in that area unfortunately. In theory the tokenization in language models shouldn’t be much different than features in tabular/imagery data. There probably are some parallels worth exploring there, I’m just not aware of any papers.

Well beware open ended questions about ai/ml research in the current “gold rush” environment. If you’re into explainability and interpretability, some folks are looking into combinatorial methods for features and their interactions to predict data coverage. This plus anthropic’s papers start to open up some new ground in interpretability for CV.

https://arxiv.org/pdf/2201.12428.pdf

What aspects? New models? Interpretability? Pipelines and scalability? Reinforcement learning? Data assurance? Too many subfields to narrow down in this question to produce a decent list, imo.

With that said, my subfield is in assurance, and some of anthropic’s work in interpretability and privileged bases is extremely interesting. Their toy models paper and the one they released last week about privileged bases in the transformer residual stream present a very novel way of thinking about model explainabity.

I apologize for how this post might come across, but your question is actually a very deep one and it will probably take a lot of up front work to get you an answer. Ai/ml is not like cinnamon- you can’t just sprinkle it on your business process and expect it to improve.

First you need to start with instrumenting your processes and building your data warehouse. Is your production flow instrumented for quality and efficiency measurement? If so, are the instruments verified? Do you have baseline performance metrics defined and expectations for improvement? Do you currently conduct any statistical process control? All of these questions have books that go with them, and we haven’t even built a trainable model yet.

I would start with some industrial engineering and applied stats textbooks and go from there. That should give you some idea of how to formulate a hypothesis and determine a method to validate it. From there you can start with the classics like an introduction to statistical learning by James et al and introduction to machine learning by alpaydin.