Viewing a single comment thread. View all comments

viperx7 OP t1_jd450pg wrote

Reply to comment by ObiWanCanShowMe in Alpaca Turbo : A chat interface to interact with alpaca models with history and context by viperx7

I added the changes required for changes but for some reason the model is painfully slow on windows

1

FermatsLastAccount t1_jd4sxbe wrote

I'm having the same issue on Linux on a 16 thread Ryzen 5850U with 64 GB of RAM.

➜  Alpaca-Turbo git:(main) python3 ./webui.py

can't load the settings file continuing with defaults
Loading Model ━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  13% -:--:--

It just stays at 13%.

1

JustAnAlpacaBot t1_jd4syhs wrote

Hello there! I am a bot raising awareness of Alpacas

Here is an Alpaca Fact:

Alpacas are healthy grazers and do not decimate natural vegetation like goats.

| Info| Code| Feedback| Contribute Fact

You don't get a fact, you earn it. If you got this fact then AlpacaBot thinks you deserved it!
0

GrapplingHobbit t1_jd98i9i wrote

Hoping you manage to figure out what is slowing things down on windows! In the direct command line interface on the 7b model the responses are almost instant for me, but pushing out around 2 minutes via Alpaca-Turbo, which is a shame because the ability to edit persona and have memory of the conversation would be great.

1

GrapplingHobbit t1_jd9c32x wrote

Also... it doesn't seem to generate at all without an internet connection. Is that the expected behaviour?

1