Skip to content

Repository files navigation

tinygrad host for iOS

Run tinygrad code using your iPhone or iPad's GPU

steps

  1. install and open this app on your iPhone or iPad
  2. install this tinygrad fork (new_ios branch) on a computer connected to the same wifi network
git clone -b new_ios https://github.com/roryclear/tinygrad.git
cd tinygrad
pip install -e .
  1. run any tinygrad code
DEV=IOS IP={your iphone/ipad IP address}:6667 PYTHONPATH=. python3.11 examples/gpt2.py --model_size=gpt2

Also try YOLOv8 on tinygrad, by caching tinygrad remote batches, models can be easily ran locally.

notes

  • Max allowed app RAM < total RAM of iOS device.
  • Metal cannot be ran in the background on iOS, the app must be open to run.

Speed

use BENCHMARK=1 to print the exec time of a graph

times in seconds, recorded on an iPhone 13.

fp32 Matmul size CoreML time Tinygrad time Tinygrad (BEAM=2) time
1x1 @ 1x1 0.00013 5.9e-06 (0.047x) 5.6e-06 (0.045x)
1x1 @ 1x4 4.9e-05 7.5e-06 (0.15x) 7.3e-06 (0.15x)
1x1 @ 1x16 4.6e-05 6.5e-06 (0.14x) 1.7e-05 (0.38x)
1x1 @ 1x64 4.8e-05 6.6e-06 (0.14x) 8.4e-06 (0.17x)
1x1 @ 1x256 4.4e-05 8e-06 (0.18x) 6.5e-06 (0.15x)
1x1 @ 1x2048 7.2e-05 8.9e-06 (0.12x) 6.3e-06 (0.088x)
1x4 @ 4x1 4.3e-05 6e-06 (0.14x) 6.5e-06 (0.15x)
1x4 @ 4x4 4.5e-05 2.2e-05 (0.49x) 7.5e-06 (0.17x)
1x4 @ 4x16 4.7e-05 8.7e-06 (0.19x) 6.2e-06 (0.13x)
1x4 @ 4x64 4.4e-05 6.8e-06 (0.15x) 3.2e-05 (0.73x)
1x4 @ 4x256 4.2e-05 7.5e-06 (0.18x) 8.6e-06 (0.2x)
1x4 @ 4x2048 4.3e-05 8.4e-06 (0.19x) 7.3e-06 (0.17x)
1x16 @ 16x1 4.3e-05 6e-06 (0.14x) 5.8e-06 (0.13x)
1x16 @ 16x4 4.7e-05 2.3e-05 (0.49x) 6.1e-06 (0.13x)
1x16 @ 16x16 4.7e-05 2.1e-05 (0.45x) 6.6e-06 (0.14x)
1x16 @ 16x64 4.5e-05 8.8e-06 (0.2x) 6e-06 (0.13x)
1x16 @ 16x256 4.5e-05 1.2e-05 (0.27x) 6.5e-06 (0.14x)
1x16 @ 16x2048 6.2e-05 1.3e-05 (0.21x) 1e-05 (0.17x)
1x64 @ 64x1 4.5e-05 7.5e-06 (0.17x) 6e-06 (0.13x)
1x64 @ 64x4 4.5e-05 7.5e-06 (0.17x) 6.1e-06 (0.14x)
1x64 @ 64x16 4.2e-05 1.1e-05 (0.25x) 6.3e-06 (0.15x)
1x64 @ 64x64 4.9e-05 1.1e-05 (0.22x) 7.4e-06 (0.15x)
1x64 @ 64x256 5.9e-05 1.7e-05 (0.29x) 1.7e-05 (0.28x)
1x64 @ 64x2048 6e-05 2.6e-05 (0.44x) 2.5e-05 (0.41x)
1x256 @ 256x1 4.4e-05 6.9e-06 (0.16x) 9.4e-06 (0.21x)
1x256 @ 256x4 4.9e-05 9.3e-06 (0.19x) 8e-06 (0.16x)
1x256 @ 256x16 4.7e-05 1.8e-05 (0.39x) 2.3e-05 (0.5x)
1x256 @ 256x64 5.1e-05 2e-05 (0.39x) 1.9e-05 (0.38x)
1x256 @ 256x256 5.3e-05 2.3e-05 (0.44x) 2.1e-05 (0.4x)
1x256 @ 256x2048 0.00016 0.00012 (0.7x) 8.6e-05 (0.52x)
1x2048 @ 2048x1 4.1e-05 1.9e-05 (0.47x) 9.7e-06 (0.24x)
1x2048 @ 2048x4 5.3e-05 2.2e-05 (0.42x) 1.1e-05 (0.21x)
1x2048 @ 2048x16 5.6e-05 0.00012 (2.2x) 2.8e-05 (0.5x)
1x2048 @ 2048x64 0.00012 0.00011 (0.95x) 4.9e-05 (0.42x)
1x2048 @ 2048x256 0.00016 0.00017 (1.1x) 8e-05 (0.49x)
1x2048 @ 2048x2048 0.0011 0.00056 (0.49x) 0.00055 (0.49x)
4x1 @ 1x1 5.9e-05 7.5e-06 (0.13x) 7.5e-06 (0.13x)
4x1 @ 1x4 4.8e-05 7e-06 (0.14x) 5.4e-06 (0.11x)
4x1 @ 1x16 4.8e-05 6.5e-06 (0.14x) 5.7e-06 (0.12x)
4x1 @ 1x64 4.9e-05 5.8e-06 (0.12x) 5.1e-06 (0.1x)
4x1 @ 1x256 4.7e-05 6e-06 (0.13x) 8e-06 (0.17x)
4x1 @ 1x2048 5.2e-05 7.5e-06 (0.15x) 6.5e-06 (0.13x)
4x4 @ 4x1 4.2e-05 6.3e-06 (0.15x) 6e-06 (0.14x)
4x4 @ 4x4 4.2e-05 6.7e-06 (0.16x) 5.4e-06 (0.13x)
4x4 @ 4x16 3.8e-05 8.5e-06 (0.22x) 6.8e-06 (0.18x)
4x4 @ 4x64 4.6e-05 5.5e-06 (0.12x) 5.8e-06 (0.13x)
4x4 @ 4x256 4.2e-05 7.6e-06 (0.18x) 1.6e-05 (0.38x)
4x4 @ 4x2048 5.5e-05 1.3e-05 (0.23x) 1.7e-05 (0.31x)
4x16 @ 16x1 4.4e-05 5.9e-06 (0.13x) 6.6e-06 (0.15x)
4x16 @ 16x4 3.9e-05 6.8e-06 (0.17x) 1.5e-05 (0.38x)
4x16 @ 16x16 4.3e-05 6.9e-06 (0.16x) 2.2e-05 (0.5x)
4x16 @ 16x64 4.8e-05 8.1e-06 (0.17x) 1.6e-05 (0.34x)
4x16 @ 16x256 4.5e-05 1.8e-05 (0.39x) 7.6e-06 (0.17x)
4x16 @ 16x2048 7e-05 2e-05 (0.28x) 1.2e-05 (0.17x)
4x64 @ 64x1 4.2e-05 6.5e-06 (0.15x) 9.1e-06 (0.22x)
4x64 @ 64x4 4.3e-05 7.1e-06 (0.16x) 8.8e-06 (0.2x)
4x64 @ 64x16 4.3e-05 7.5e-06 (0.17x) 2.1e-05 (0.48x)
4x64 @ 64x64 4.6e-05 1.6e-05 (0.34x) 1.2e-05 (0.26x)
4x64 @ 64x256 4.8e-05 2.8e-05 (0.58x) 1.5e-05 (0.3x)
4x64 @ 64x2048 6.6e-05 3.3e-05 (0.5x) 3.2e-05 (0.48x)
4x256 @ 256x1 4.1e-05 8.1e-06 (0.2x) 6.7e-06 (0.16x)
4x256 @ 256x4 3.9e-05 1.1e-05 (0.27x) 7.7e-06 (0.2x)
4x256 @ 256x16 4.5e-05 1.2e-05 (0.27x) 1.1e-05 (0.24x)
4x256 @ 256x64 4.8e-05 3.6e-05 (0.75x) 1.9e-05 (0.39x)
4x256 @ 256x256 5.3e-05 0.00015 (2.8x) 3.7e-05 (0.7x)
4x256 @ 256x2048 0.00018 0.00034 (1.9x) 0.0001 (0.57x)
4x2048 @ 2048x1 5.2e-05 1.8e-05 (0.34x) 1.1e-05 (0.2x)
4x2048 @ 2048x4 5e-05 3.4e-05 (0.68x) 3.2e-05 (0.64x)
4x2048 @ 2048x16 5.3e-05 8.6e-05 (1.6x) 3.9e-05 (0.74x)
4x2048 @ 2048x64 6.6e-05 0.00039 (5.9x) 6.3e-05 (0.95x)
4x2048 @ 2048x256 0.00017 0.0017 (9.7x) 0.00014 (0.81x)
4x2048 @ 2048x2048 0.0016 0.00063 (0.41x) 0.0015 (0.93x)
16x1 @ 1x1 5.5e-05 6.1e-06 (0.11x) 6.8e-06 (0.12x)
16x1 @ 1x4 4.3e-05 9e-06 (0.21x) 2e-05 (0.47x)
16x1 @ 1x16 4.4e-05 6.1e-06 (0.14x) 5.8e-06 (0.13x)
16x1 @ 1x64 4.7e-05 1.5e-05 (0.32x) 5.9e-06 (0.13x)
16x1 @ 1x256 4.6e-05 7e-06 (0.15x) 5.8e-06 (0.13x)
16x1 @ 1x2048 5.6e-05 9.8e-06 (0.17x) 1.2e-05 (0.21x)
16x4 @ 4x1 4.6e-05 1.7e-05 (0.37x) 1e-05 (0.22x)
16x4 @ 4x4 4.4e-05 5.9e-06 (0.13x) 5.8e-06 (0.13x)
16x4 @ 4x16 4.2e-05 1.8e-05 (0.44x) 9.4e-06 (0.22x)
16x4 @ 4x64 4.8e-05 6e-06 (0.13x) 1e-05 (0.21x)
16x4 @ 4x256 4.6e-05 7.7e-06 (0.17x) 1.5e-05 (0.32x)
16x4 @ 4x2048 6.2e-05 1.1e-05 (0.18x) 2.2e-05 (0.36x)
16x16 @ 16x1 4.2e-05 6.6e-06 (0.16x) 7.2e-06 (0.17x)
16x16 @ 16x4 4.1e-05 6.4e-06 (0.16x) 6.2e-06 (0.15x)
16x16 @ 16x16 4.1e-05 8.3e-06 (0.2x) 6.6e-06 (0.16x)
16x16 @ 16x64 4.1e-05 1.6e-05 (0.4x) 6.8e-06 (0.17x)
16x16 @ 16x256 5.2e-05 1.8e-05 (0.35x) 8.6e-06 (0.17x)
16x16 @ 16x2048 6.5e-05 2.4e-05 (0.37x) 2.5e-05 (0.38x)
16x64 @ 64x1 4.1e-05 5.9e-06 (0.14x) 7.2e-06 (0.18x)
16x64 @ 64x4 4.1e-05 1.7e-05 (0.41x) 1.9e-05 (0.46x)
16x64 @ 64x16 4.5e-05 1.9e-05 (0.43x) 1.1e-05 (0.25x)
16x64 @ 64x64 4.4e-05 2.9e-05 (0.67x) 1.3e-05 (0.29x)
16x64 @ 64x256 5e-05 3.8e-05 (0.76x) 2e-05 (0.4x)
16x64 @ 64x2048 6.1e-05 4.2e-05 (0.69x) 7.7e-05 (1.3x)
16x256 @ 256x1 4.7e-05 7.9e-06 (0.17x) 1.1e-05 (0.23x)
16x256 @ 256x4 4.8e-05 1.6e-05 (0.34x) 2e-05 (0.42x)
16x256 @ 256x16 5e-05 3.7e-05 (0.74x) 1e-05 (0.2x)
16x256 @ 256x64 5.1e-05 0.00014 (2.8x) 1.7e-05 (0.33x)
16x256 @ 256x256 5.3e-05 7.3e-05 (1.4x) 5.4e-05 (1x)
16x256 @ 256x2048 0.00017 0.00019 (1.1x) 0.00023 (1.4x)
16x2048 @ 2048x1 5.8e-05 1.9e-05 (0.32x) 1.6e-05 (0.27x)
16x2048 @ 2048x4 5.5e-05 4.7e-05 (0.85x) 4.1e-05 (0.75x)
16x2048 @ 2048x16 6.1e-05 0.00042 (6.9x) 3e-05 (0.49x)
16x2048 @ 2048x64 6.7e-05 0.0017 (25x) 7.2e-05 (1.1x)
16x2048 @ 2048x256 0.00018 0.00048 (2.6x) 0.00031 (1.7x)
16x2048 @ 2048x2048 0.0015 0.0015 (1x) 0.0013 (0.88x)
64x1 @ 1x1 4.9e-05 6.1e-06 (0.12x) 7.4e-06 (0.15x)
64x1 @ 1x4 4.5e-05 1.2e-05 (0.27x) 5.9e-06 (0.13x)
64x1 @ 1x16 4.4e-05 6.8e-06 (0.16x) 5.8e-06 (0.13x)
64x1 @ 1x64 4.9e-05 7.5e-06 (0.15x) 8.6e-06 (0.18x)
64x1 @ 1x256 4.5e-05 9.3e-06 (0.21x) 8.6e-06 (0.19x)
64x1 @ 1x2048 0.00011 3.2e-05 (0.28x) 1.7e-05 (0.15x)
64x4 @ 4x1 4.8e-05 7e-06 (0.14x) 7.6e-06 (0.16x)
64x4 @ 4x4 4.5e-05 6.5e-06 (0.15x) 1.9e-05 (0.43x)
64x4 @ 4x16 4.9e-05 1.6e-05 (0.33x) 5.9e-06 (0.12x)
64x4 @ 4x64 4.5e-05 9.1e-06 (0.2x) 7.8e-06 (0.17x)
64x4 @ 4x256 5.2e-05 1.1e-05 (0.21x) 2.1e-05 (0.41x)
64x4 @ 4x2048 8.4e-05 3.6e-05 (0.42x) 2.4e-05 (0.29x)
64x16 @ 16x1 4.1e-05 1.9e-05 (0.47x) 6.7e-06 (0.16x)
64x16 @ 16x4 4e-05 8.8e-06 (0.22x) 7.1e-06 (0.18x)
64x16 @ 16x16 4.1e-05 1.6e-05 (0.4x) 1.6e-05 (0.39x)
64x16 @ 16x64 4.2e-05 2.1e-05 (0.49x) 1e-05 (0.25x)
64x16 @ 16x256 4.8e-05 3e-05 (0.64x) 1.8e-05 (0.38x)
64x16 @ 16x2048 0.0001 6.1e-05 (0.6x) 6.2e-05 (0.61x)
64x64 @ 64x1 4.8e-05 8.9e-06 (0.18x) 7.7e-06 (0.16x)
64x64 @ 64x4 4.3e-05 1.1e-05 (0.25x) 1.9e-05 (0.44x)
64x64 @ 64x16 4.2e-05 3.3e-05 (0.78x) 2.1e-05 (0.5x)
64x64 @ 64x64 4.7e-05 2.6e-05 (0.55x) 2e-05 (0.42x)
64x64 @ 64x256 5.1e-05 4.6e-05 (0.9x) 4.9e-05 (0.97x)
64x64 @ 64x2048 7.9e-05 0.00014 (1.8x) 0.0003 (3.7x)
64x256 @ 256x1 4.6e-05 1.5e-05 (0.33x) 1.1e-05 (0.23x)
64x256 @ 256x4 5.2e-05 4e-05 (0.76x) 1.2e-05 (0.22x)
64x256 @ 256x16 5e-05 0.00017 (3.3x) 1.8e-05 (0.36x)
64x256 @ 256x64 5.2e-05 7.5e-05 (1.4x) 3.5e-05 (0.68x)
64x256 @ 256x256 5.4e-05 8.8e-05 (1.6x) 0.0001 (1.9x)
64x256 @ 256x2048 0.00018 0.00049 (2.8x) 0.00056 (3.1x)
64x2048 @ 2048x1 7.5e-05 3.5e-05 (0.47x) 2.7e-05 (0.36x)
64x2048 @ 2048x4 8e-05 0.00015 (1.9x) 4.3e-05 (0.54x)
64x2048 @ 2048x16 8.8e-05 0.0017 (19x) 7.6e-05 (0.87x)
64x2048 @ 2048x64 8.5e-05 0.0006 (7.1x) 0.00027 (3.1x)
64x2048 @ 2048x256 0.00023 0.0007 (3x) 0.00077 (3.3x)
64x2048 @ 2048x2048 0.0027 0.0039 (1.4x) 0.0041 (1.5x)
256x1 @ 1x1 4.9e-05 7.5e-06 (0.15x) 7.9e-06 (0.16x)
256x1 @ 1x4 4.3e-05 7e-06 (0.16x) 8.5e-06 (0.2x)
256x1 @ 1x16 4.4e-05 7.8e-06 (0.18x) 6.7e-06 (0.15x)
256x1 @ 1x64 4.6e-05 9.7e-06 (0.21x) 7e-06 (0.15x)
256x1 @ 1x256 6.2e-05 1.9e-05 (0.31x) 1.4e-05 (0.22x)
256x1 @ 1x2048 0.00015 0.00011 (0.73x) 7.9e-05 (0.53x)
256x4 @ 4x1 4e-05 7.9e-06 (0.2x) 7.2e-06 (0.18x)
256x4 @ 4x4 4.2e-05 7.5e-06 (0.18x) 7e-06 (0.17x)
256x4 @ 4x16 4.8e-05 1e-05 (0.21x) 6.4e-06 (0.13x)
256x4 @ 4x64 4.7e-05 1.2e-05 (0.25x) 1.1e-05 (0.23x)
256x4 @ 4x256 5.8e-05 2.1e-05 (0.35x) 2.2e-05 (0.38x)
256x4 @ 4x2048 0.00017 0.00012 (0.71x) 0.00012 (0.69x)
256x16 @ 16x1 4.8e-05 8.5e-06 (0.18x) 7.5e-06 (0.16x)
256x16 @ 16x4 4.7e-05 2.7e-05 (0.58x) 1.1e-05 (0.23x)
256x16 @ 16x16 5.1e-05 1.6e-05 (0.31x) 8.4e-06 (0.16x)
256x16 @ 16x64 5.3e-05 1.7e-05 (0.32x) 2.3e-05 (0.44x)
256x16 @ 16x256 6.7e-05 3.7e-05 (0.55x) 4.1e-05 (0.61x)
256x16 @ 16x2048 0.00024 0.0002 (0.84x) 0.00038 (1.6x)
256x64 @ 64x1 4.5e-05 9.3e-06 (0.21x) 9e-06 (0.2x)
256x64 @ 64x4 4.6e-05 2.8e-05 (0.6x) 2.2e-05 (0.48x)
256x64 @ 64x16 5.4e-05 3.5e-05 (0.65x) 1.7e-05 (0.32x)
256x64 @ 64x64 4.9e-05 3.9e-05 (0.8x) 5.3e-05 (1.1x)
256x64 @ 64x256 6e-05 7.5e-05 (1.3x) 0.00013 (2.2x)
256x64 @ 64x2048 0.00016 0.00051 (3.1x) 0.00053 (3.2x)
256x256 @ 256x1 6.4e-05 2.1e-05 (0.33x) 1.7e-05 (0.27x)
256x256 @ 256x4 6.1e-05 8.1e-05 (1.3x) 5.5e-05 (0.9x)
256x256 @ 256x16 7.1e-05 9.6e-05 (1.4x) 5.6e-05 (0.78x)
256x256 @ 256x64 5.9e-05 8.9e-05 (1.5x) 0.00016 (2.7x)
256x256 @ 256x256 7e-05 0.00026 (3.7x) 0.00028 (4x)
256x256 @ 256x2048 0.00035 0.0016 (4.5x) 0.0029 (8.3x)
256x2048 @ 2048x1 0.00025 0.0001 (0.4x) 8.4e-05 (0.33x)
256x2048 @ 2048x4 0.00025 0.00057 (2.3x) 0.00026 (1.1x)
256x2048 @ 2048x16 0.00028 0.00072 (2.6x) 0.00052 (1.8x)
256x2048 @ 2048x64 0.00028 0.00061 (2.1x) 0.00073 (2.6x)
256x2048 @ 2048x256 0.0011 0.0021 (1.9x) 0.0029 (2.7x)
256x2048 @ 2048x2048 0.0057 0.0069 (1.2x) 0.011 (1.9x)
2048x1 @ 1x1 5e-05 8e-06 (0.16x) 9.2e-06 (0.18x)
2048x1 @ 1x4 5.4e-05 8.3e-06 (0.15x) 6.6e-06 (0.12x)
2048x1 @ 1x16 5.3e-05 1.3e-05 (0.25x) 7.9e-06 (0.15x)
2048x1 @ 1x64 7.3e-05 3.1e-05 (0.43x) 2.6e-05 (0.35x)
2048x1 @ 1x256 0.00016 0.00011 (0.7x) 7.3e-05 (0.47x)
2048x1 @ 1x2048 0.0024 0.00092 (0.38x) 0.0007 (0.29x)
2048x4 @ 4x1 5.2e-05 8.8e-06 (0.17x) 8.1e-06 (0.16x)
2048x4 @ 4x4 5.3e-05 2.6e-05 (0.48x) 9.2e-06 (0.17x)
2048x4 @ 4x16 5.4e-05 1.5e-05 (0.28x) 1.2e-05 (0.21x)
2048x4 @ 4x64 8e-05 3.6e-05 (0.45x) 3.1e-05 (0.39x)
2048x4 @ 4x256 0.00017 0.00012 (0.74x) 7.9e-05 (0.48x)
2048x4 @ 4x2048 0.0024 0.001 (0.41x) 0.00079 (0.33x)
2048x16 @ 16x1 5.6e-05 2.7e-05 (0.48x) 1.2e-05 (0.21x)
2048x16 @ 16x4 5.9e-05 2.6e-05 (0.44x) 1.8e-05 (0.3x)
2048x16 @ 16x16 6.4e-05 3.4e-05 (0.53x) 1.9e-05 (0.3x)
2048x16 @ 16x64 0.0001 6e-05 (0.6x) 0.00011 (1.1x)
2048x16 @ 16x256 0.00023 0.0002 (0.87x) 0.00035 (1.5x)
2048x16 @ 16x2048 0.0028 0.0015 (0.53x) 0.001 (0.38x)
2048x64 @ 64x1 8e-05 3e-05 (0.37x) 2.5e-05 (0.31x)
2048x64 @ 64x4 8.1e-05 5e-05 (0.62x) 6.5e-05 (0.8x)
2048x64 @ 64x16 9.9e-05 9.9e-05 (1x) 0.00013 (1.3x)
2048x64 @ 64x64 7.6e-05 0.00014 (1.8x) 0.00035 (4.6x)
2048x64 @ 64x256 0.00015 0.0005 (3.3x) 0.00087 (5.7x)
2048x64 @ 64x2048 0.0024 0.0039 (1.7x) 0.0045 (1.9x)
2048x256 @ 256x1 0.00026 0.00012 (0.44x) 8e-05 (0.3x)
2048x256 @ 256x4 0.00025 0.00013 (0.53x) 0.00015 (0.61x)
2048x256 @ 256x16 0.00029 0.00038 (1.3x) 0.00024 (0.81x)
2048x256 @ 256x64 0.00023 0.00049 (2.2x) 0.00075 (3.3x)
2048x256 @ 256x256 0.00042 0.0019 (4.5x) 0.0029 (6.9x)
2048x256 @ 256x2048 0.0089 0.012 (1.3x) 0.0096 (1.1x)
2048x2048 @ 2048x1 0.0017 0.0011 (0.66x) 0.00056 (0.33x)
2048x2048 @ 2048x4 0.0018 0.00095 (0.54x) 0.00097 (0.55x)
2048x2048 @ 2048x16 0.0019 0.003 (1.6x) 0.0026 (1.4x)
2048x2048 @ 2048x64 0.0035 0.0039 (1.1x) 0.0034 (0.99x)
2048x2048 @ 2048x256 0.012 0.008 (0.67x) 0.02 (1.7x)
2048x2048 @ 2048x2048 0.021 0.036 (1.7x) 0.034 (1.6x)

About

run tinygrad on your iphone over a network

Topics

Resources

Stars

8 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages