Viewing a single comment thread. View all comments

visarga t1_izvv9xi wrote

I'd be interested in knowing, too. I want to parse the HTML of a page and identify what actions are possible, such as modifying text in an input or clicking a button. But web pages often go over 30K tokens so there's no way to fit them. HTML can be extremely verbose.

2

Acceptable-Cress-374 t1_izw9kw9 wrote

Would it help to parse the html with "regular" code (there are a bunch of python libs that do this) and give that parsed list to GPT3?

3

asmallstep t1_j01daaa wrote

Any libraries that you can recommend for this task?
It doesn't sound like a trivial problem. For example, a label "Email" above a field called "input1" would add relevant information that seems hard to parse. Taking the layout into account would be really helpful though.

2

granddaddy OP t1_izyu345 wrote

this sounds like the right answer (and sth i need to keep in mind as well)

just as an FYI, this is one answer i found from a twitter thread

  • the data that needs to be fed into the model is divided into chunks
  • when a user asks a question, each of these chunks (likely less than 4k tokens) is reviewed
  • when there is a section of the chunk that is relevant, that section is combined with the user question
  • this combined text is fed as prompt, and GPT-3 is able to answer the user's question

there's a prebuilt openai notebook you can use to replicate it

1