TI-84 PLUS + ESP32-S3 + OV5640 AI CALCULATOR
BUILDER SPECIFICATION
1. PROJECT OVERVIEW
Build a modified TI-84 Plus non-CE calculator with an internally installed Seeed Studio XIAO ESP32-S3 Sense, OV5640 5MP autofocus camera, Wi-Fi, OpenAI API connectivity, TI-84 to ESP32 communication, and a calculator-side AI interface.
System architecture:
OV5640 camera -> ESP32-S3 -> Wi-Fi -> OpenAI API -> ESP32-S3 -> TI-84 -> Display
The ESP32-S3 handles camera control, image processing, networking, and AI communication. The TI-84 provides the keyboard, display, user interface, and commands.
Use the existing TI84-AI GitHub project by ivan-grebe as the primary software and architecture reference:
https://github.com/ivan-grebe/ti84-ai
Do not simply adapt an ESP32-C3 tutorial. This build specifically uses an ESP32-S3, OV5640, and TI-84 Plus.
2. HARDWARE
Required hardware:
* TI-84 Plus non-CE, fully functioning before modification
* Seeed Studio XIAO ESP32-S3 Sense, preferably with sufficient PSRAM
* OV5640 5MP autofocus camera compatible with the XIAO ESP32-S3 Sense
* Thin insulated hookup wire
* Appropriate soldering equipment
* Heat-shrink and insulation
* Mechanical mounting materials
* USB-C cable
* Computer for firmware development and flashing
Do not permanently close the calculator until all electronics and software have been tested.
3. SYSTEM RESPONSIBILITIES
ESP32-S3 responsibilities:
* Camera control and image capture
* JPEG compression
* Base64 encoding
* Wi-Fi connection
* HTTPS communication
* OpenAI API communication
* AI request and response handling
TI-84 responsibilities:
* Keyboard input
* Basic user interface
* Sending commands to the ESP32
* Displaying AI responses
* Scrolling through long responses
Preferred image pipeline:
OV5640 -> frame buffer -> JPEG -> Base64 -> HTTPS/OpenAI
Optimize image resolution, JPEG quality, PSRAM usage, RAM usage, and upload speed. Use a balanced image-quality setting by default, with higher resolution available when small text or equations need to be recognized.
4. TI-84 TO ESP32 COMMUNICATION
Use the TI-84's existing 2.5 mm link-port interface for communication, following the established TI84-AI implementation.
Reference signal mapping:
Tip -> ESP32-S3 D0
Ring -> ESP32-S3 D1
Verify the exact TI-84 board revision, electrical characteristics, voltage levels, and pin functions before soldering.
Do not connect unknown signals without verifying their voltage levels and functions.
Communication must support:
* TI-84 to ESP32 commands
* Text transmission
* Camera commands
* ESP32 to TI-84 responses
* Status and error messages
* Long responses
5. ESP32-S3 FIRMWARE
Use the TI84-AI project as the initial software foundation.
Development should use the project's documented environment, including Python, Git, VS Code, and PlatformIO.
Compile firmware specifically for the ESP32-S3, not ESP32-C3.
Firmware requirements:
Wi-Fi Manager:
* Configure Wi-Fi SSID
* Configure Wi-Fi password
* Automatically reconnect after temporary Wi-Fi loss
OpenAI Configuration:
* Configure the user's OpenAI API credentials
* Never hard-code API keys into publicly distributed source code
* Make the AI model configurable
Camera Configuration:
* Allow supported image resolution and quality settings to be changed
AI Communication:
1. Capture the camera image.
2. Compress the image to JPEG.
3. Encode the image appropriately.
4. Send the request over HTTPS to the OpenAI API.
5. Receive the response.
6. Parse the response.
7. Send the answer back to the TI-84.
Handle Wi-Fi failures, API timeouts, authentication failures, invalid API responses, oversized images, network failures, and empty responses.
6. CAMERA
Use the OV5640 5MP autofocus camera.
Mount the camera securely inside the calculator with a clear forward-facing view.
Test:
* Autofocus
* Image orientation
* Field of view
* Lighting
* Image sharpness
* Text readability
* Equation readability
The camera must not move during normal handling.
If necessary, create a small optical opening or window in the calculator shell.
Finalize the camera position only after testing the live image.
7. CALCULATOR-SIDE AI PROGRAM
Base the calculator-side program on the existing TI84-AI TI-84 software.
Text questions:
The user enters a question using the TI-84 keyboard. The calculator sends it to the ESP32. The ESP32 sends it to OpenAI. The response is returned and displayed on the TI-84.
Camera questions:
The calculator sends a camera command. The ESP32 captures an image with the OV5640. The image is sent to the AI model. The response is returned to the TI-84.
Long responses must be scrollable using the TI-84 keypad.
Display understandable error messages such as:
* Wi-Fi unavailable
* API error
* Camera error
* No response
* Configuration required
8. REQUIRED CUSTOM LAUNCHER SEQUENCE
The calculator must support this exact launcher sequence.
Step 1:
5 -> STOP -> ALPHA -> C -> ENTER
Step 2:
2nd -> CATALOG -> LN
Step 3:
Scroll down to:
Send(
Press ENTER.
Step 4:
ALPHA -> C -> ENTER
Result:
After completing this sequence, the AI application should be accessible through the calculator's normal PRGM menu.
Implement this using an appropriate TI-84 program or assembly launcher mechanism.
This exact sequence must be tested on the actual TI-84 Plus hardware.
Do not assume that Send( will work simply because it appears in the calculator's catalog.
The launcher must not interfere with:
* Normal calculator operation
* Existing programs
* Calculator resets
* Normal PRGM operation
* Keypad functionality
9. RESET AND RAM BEHAVIOR
A normal TI-84 RAM reset should return the calculator to a clean normal calculator state while the ESP32-S3 hardware and firmware remain installed.
The system must not depend on temporary calculator RAM contents remaining intact.
Test:
* RAM reset
* Calculator power cycle
* Battery removal and reinstallation
* ESP32 reboot
* Wi-Fi loss and reconnection
After restarting, the user must be able to invoke the designated launcher again and use the AI normally.
10. CONFIGURATION SYSTEM
Provide a simple configuration system, preferably based on the TI84-AI project's configuration portal.
Configuration options:
* Wi-Fi SSID
* Wi-Fi password
* OpenAI API key
* AI model
* Camera resolution
* JPEG quality
* Optional debugging
API credentials must not be exposed in public source code.
11. PHYSICAL INSTALLATION
Install the ESP32-S3 and camera internally while maintaining normal calculator usability.
Ensure:
* LCD remains unobstructed
* Keypad works normally
* Battery compartment remains accessible
* Calculator shell closes correctly
* Wires are not pinched
* Components do not interfere with buttons
* Camera is firmly mounted
* No exposed conductive connections can short
* Calculator remains serviceable
* ESP32 placement minimizes interference with the calculator PCB
12. POWER SYSTEM
Determine and verify the appropriate power source and regulation for the ESP32-S3 and OV5640.
Before final installation:
* Measure supply voltage
* Verify current requirements
* Verify startup current
* Confirm the calculator power circuitry is not overloaded
* Check for excessive heat
* Test at different battery charge levels
Do not assume the TI-84 power rail can safely power the ESP32-S3 and OV5640 without appropriate verification or regulation.
13. TESTING PROCEDURE
Stage 1: ESP32
Before modifying the calculator:
* Flash firmware
* Confirm ESP32 boots
* Confirm Wi-Fi
* Confirm camera
* Confirm OV5640 autofocus
* Confirm JPEG capture
* Confirm OpenAI API communication
Stage 2: Camera
Test:
* Live image
* Focus
* Orientation
* Lighting
* Text
* Mathematical equations
Stage 3: Calculator Communication
Test:
* TI-84 to ESP32 communication
* ESP32 to TI-84 communication
* Test text transmission
* Test errors
* Test long responses
Stage 4: AI
Test:
* Text questions
* Camera questions
* Image recognition
* Mathematical problems
* Follow-up questions
* Long answers
Stage 5: Launcher
Test the complete exact sequence:
5 -> STOP -> ALPHA -> C -> ENTER
then:
2nd -> CATALOG -> LN -> Send( -> ENTER -> ALPHA -> C -> ENTER
Confirm that the AI application can subsequently be accessed through the normal PRGM interface.
Stage 6: Reset
Test:
* RAM reset
* Power cycle
* Battery removal
* ESP32 restart
* Wi-Fi loss and reconnection
Stage 7: Final Assembly
Only after all tests pass:
* Secure ESP32
* Secure camera
* Insulate connections
* Close calculator shell
* Perform a complete final test
14. FINAL ACCEPTANCE CRITERIA
The completed calculator must:
* Remain a functioning TI-84 Plus
* Contain the ESP32-S3 internally
* Contain the OV5640 5MP autofocus camera
* Connect to Wi-Fi
* Communicate with OpenAI
* Capture camera images
* Send image questions to the AI
* Answer text questions
* Display AI responses
* Scroll long responses
* Support the exact launcher sequence
* Continue working after calculator RAM resets
* Not interfere with normal calculator functionality
* Have no exposed unsafe electrical connections
* Allow firmware updates without rebuilding the entire calculator
15. BUILDER DELIVERABLES
Provide:
1. Fully assembled modified TI-84 Plus
2. ESP32-S3 firmware source
3. TI-84 program source
4. Wi-Fi configuration instructions
5. OpenAI API key configuration instructions
6. Instructions for changing the AI model
7. Camera testing procedure
8. Exact launcher sequence instructions
9. Reset and recovery instructions
10. Complete hardware component list
11. Wiring and pin documentation
12. Basic troubleshooting guide
PRIMARY REFERENCE
TI84-AI GitHub:
https://github.com/ivan-grebe/ti84-ai
Review its ESP32 firmware, TI-84 program, link-port implementation, camera implementation, Wi-Fi configuration, OpenAI communication, and build instructions.
Adapt the reference specifically for the ESP32-S3, OV5640, and TI-84 Plus architecture.
... Show more