“Napkinmatic Ubiquitous Pocket AGI: Utility-context-sensitive 3D AR XR HCI for Vision to LLM”
Conference:
Type(s):
Title:
- Napkinmatic Ubiquitous Pocket AGI: Utility-context-sensitive 3D AR XR HCI for Vision to LLM
Developer(s):
Description:
This app lets you connect your LLM to the real world via both converting vision to semantic data to directly interfacing with your vision-based LLM and loading context-sensitive menus. Example: Scan any diagram to be able to understand it and to load its corresponding 3D model (or AI3D generate it).
References:
[1]
Yosun Chang. 2020a. Special Interest Group on Computer Graphics and Interactive Techniques Conference Real-Time Live! (SIGGRAPH ?20 Real-Time Live!) (Aug. 2020). https://doi.org/10.1145/3407662.3407755
[2]
Yosun Chang. 2020b. DrawmaticAR: automagical AR content from written words!. In ACM SIGGRAPH 2020 Real-Time Live! (Virtual Event, USA) (SIGGRAPH ?20). Association for Computing Machinery, New York, NY, USA, Article 3, 1 pages. https://doi.org/10.1145/3407662.3407755
[3]
Yosun Chang. 2023a. #EmadAI via SMBDY.ai. https://x.com/Yosun/status/1735798618328584438.
[4]
Yosun Chang. 2023b. Napkinmatic. In ACM SIGGRAPH 2023 Appy Hour (Los Angeles, CA, USA) (SIGGRAPH ?23). Association for Computing Machinery, New York, NY, USA, Article 4, 2 pages. https://doi.org/10.1145/3588427.3595357
[5]
Yosun Chang. 2023c. Napkinmatic. In ACM SIGGRAPH 2023 Appy Hour (Los Angeles, CA, USA) (SIGGRAPH ?23). Association for Computing Machinery, New York, NY, USA, Article 4, 2 pages. https://doi.org/10.1145/3588427.3595357
[6]
Yosun Chang. 2023d. Napkinmatic – Open Framework. Retrieved May 23, 2023 from https://github.com/yosun/napkinmatic
[7]
Yosun Chang. 2023e. Napkinmatic 3D. In ACM SIGGRAPH 2023 Real-Time Live! (Los Angeles, CA, USA) (SIGGRAPH ?23). Association for Computing Machinery, New York, NY, USA, Article 6, 2 pages. https://doi.org/10.1145/3588430.3597253
[8]
Yosun Chang. 2023f. Napkinmatic TED AI expo demo. https://x.com/Yosun/status/1714437768502534423.
[9]
Yosun Chang. 2023g. Napkinmatic3D Toy Ripper Mode. https://x.com/Yosun/status/1721892003041149434.
[10]
Yosun Chang. 2023h. PocketAGI Hackathon App. https://devpost.com/software/pocketagi.
[11]
Yosun Chang. 2024a. AI3D Camera Obscura. https://x.com/Yosun/status/1774521401355166148/.
[12]
Yosun Chang. 2024b. VolumeMatic and the AI3D.foundation: Text to interactive volume app spatial computing creator app. SIGGRAPH ?24: ACM SIGGRAPH 2024 Appy Hour (2024). https://dl.acm.org/doi/10.1145/3664294.3664361
[13]
Ludwiczuk Bartosz Lavreniuk, Mykola. 2024. StableDesign. https://github.com/Lavreniuk/generative-interior-design.
[14]
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding Conditional Control to Text-to-Image Diffusion Models.
