unicodemonkey
unicodemonkey t1_j2auxih wrote
Hi. Thanks for the code, I've used 7HWRPY9RXEWY.
The app does work for me even with a fairly large index (35K photos) and I have some feedback to share:
- a first-time user can type in a query before being asked to build the index. Might be better to offer indexing right after the first start.
- the query doesn't get re-run automatically after indexing completes, so the user sees the "no index, no results" response to the initial query until they try searching again
- the indexer has to rely on low-res thumbnails when processing photos that have been offloaded to iCloud. Does this affect accuracy? I'm not sure if there are enough pixels for CLIP.
- such photos don't get redownloaded from iCloud when I'm viewing them in the search results. I just get blurry thumbnails.
- there's no way to actually do anything useful with a search result. The "Share" button would be a welcome addition, as well as metadata display and a viewer that supports the zoom gesture.
- I see you l've extended the number of search results from 12 to 120, great. Maybe it's possible to load more results dynamically when scrolling instead of a configurable hard limit.
- I think ranking just by similarity is not intuitive enough, though. Recent photos or favorites are likely to be more important for the user, for example. Just an idea for future improvement - a simple ranking model over CLIP similarity and a number of other features might be useful.
- Would be nice to have search restricted by a particular album
- The model does produce unexpected results at times - e.g. "orange cat" seems to be a fitting description for a gray cat sitting on an orange blanket.
unicodemonkey t1_j2bdf3g wrote
Reply to comment by RingoCatKeeper in [P]Run CLIP on your iPhone to Search Photos offline. by RingoCatKeeper
I think network access would be legitimate if used specifically by the iCloud service to display photos. It probably happens in a separate background process that manages the photo library, not in the app itself. But it's up to you to decide, of course.