Overview
In this tutorial, you will build an Android app that turns an ordinary photo into a 3D landscape. Bright areas of the image rise into hills, dark areas sink into valleys, and the result is drawn directly on the GPU using the AndroidX WebGPU library and Jetpack Compose.
How it works
The whole technique rests on one rule: the brighter a pixel is, the higher it sits. The app processes an image in five stages:
- The user picks an image from the device.
- The image is resized to a 64 × 64 grid.
- The brightness of each pixel is calculated.
- Those values are stored in a height map.
- A shader reads the height map and draws a 3D-looking surface.
Two small GPU programs, called shaders and written in WGSL (WebGPU Shading Language), do the drawing:
- The vertex shader decides where each point of the surface appears on screen.
- The fragment shader decides what color each pixel should be.
Project files
The app is split into three files, each with a single responsibility:
WebGpuRenderer.kt
Image processing, shaders, and GPU drawing
WebGpuSurface.kt
Connects Compose to the renderer and manages the drawing surface
MainActivity.kt
The UI: pick an image, preview it, and tap Generate 3D
Prerequisites
- Android Studio with Jetpack Compose support
- Basic knowledge of Kotlin and Compose
- A physical Android device (API 26 or higher) with a modern GPU
No previous graphics programming experience is required.
Step 1: Create the Project and Add Dependencies
- In Android Studio, select File → New → New Project and choose the Empty Activity template.
- Set the Minimum SDK to API 26.
- Open your module’s build.gradle.kts and add the WebGPU library and Coil (used for the image preview):
dependencies {
implementation("androidx.webgpu:webgpu:1.0.0-alpha06")
implementation("io.coil-kt:coil-compose:<latest-version>")
}
- Sync the project.
- Create two new Kotlin files next to MainActivity.kt: WebGpuRenderer.kt and WebGpuSurface.kt.
Tip: the WebGPU library is still in alpha. Check the AndroidX WebGPU release page for the current version before you start.
Step 2: Create the Renderer Class and Initialize WebGPU
Open WebGpuRenderer.kt and start with the class, its constants, and its state:
/**
* Image-to-3D relief renderer.
*
* Bright pixels produce higher areas.
* Dark pixels produce lower areas.
*/
class WebGpuRenderer(
private val context: Context
) {
companion object {
private const val TAG = "WebGpuRenderer"
private const val GRID_SIZE = 64
private const val VERTEX_COUNT =
(GRID_SIZE - 1) * (GRID_SIZE - 1) * 6
}
private var webGpu: WebGpu? = null
private var renderPipeline: GPURenderPipeline? = null
// Default height map: flat surface.
private var heightMap = FloatArray(GRID_SIZE * GRID_SIZE) { 0.5f }
What these values mean:
- GRID_SIZE = 64 gives a 64 × 64 grid, or 4,096 height values.
- VERTEX_COUNT is the number of vertices to draw. The grid has 63 × 63 squares, each made of two triangles (six vertices), for a total of 23,814.
- heightMap starts at 0.5f everywhere, so before any image is loaded the renderer draws a flat plane.
Next, add the init() function. It creates the WebGPU instance for the Android Surface and configures where and how to draw:
suspend fun init(surface: Surface, width: Int, height: Int) {
require(width > 0 && height > 0) {
"Surface dimensions must be positive"
}
webGpu?.close()
val gpu = createWebGpu(surface)
webGpu = gpu
gpu.webgpuSurface.configure(
GPUSurfaceConfiguration(
gpu.device,
width,
height,
TextureFormat.RGBA8Unorm
)
)
createPipeline()
Log.d(TAG, "WebGPU initialized: ${width}x${height}")
}
createWebGpu(surface) gives you a device (your connection to the GPU) and a surface (the area of the screen to draw into). configure() sets the surface size and its pixel format, RGBA8Unorm. You will write createPipeline() in Step 4.
Step 3: Convert the Image into a Height Map
Still inside WebGpuRenderer, add setImageUri(). It loads the selected image, shrinks it to the grid size, and converts every pixel into a height value between 0 and 1:
/**
* Loads an image and converts its brightness to vertex heights.
* Call after init() once the GPU device is ready.
*/
fun setImageUri(uri: Uri) {
try {
val bitmap = context.contentResolver
.openInputStream(uri)
.use { stream ->
requireNotNull(stream) { "Unable to open selected image" }
BitmapFactory.decodeStream(stream)
} ?: error("Unable to decode selected image")
val scaled = Bitmap.createScaledBitmap(
bitmap, GRID_SIZE, GRID_SIZE, true
)
val newHeights = FloatArray(GRID_SIZE * GRID_SIZE)
for (y in 0 until GRID_SIZE) {
for (x in 0 until GRID_SIZE) {
val pixel = scaled.getPixel(x, y)
val red = (pixel shr 16) and 0xFF
val green = (pixel shr 8) and 0xFF
val blue = pixel and 0xFF
// Perceived brightness (BT.601 luma), normalized to 0..1.
val luminance = (
0.299f * red +
0.587f * green +
0.114f * blue
) / 255f
newHeights[y * GRID_SIZE + x] = luminance
}
}
heightMap = newHeights
if (scaled !== bitmap) {
scaled.recycle()
}
bitmap.recycle()
// Rebuild the shader with the new height map.
if (webGpu != null) {
createPipeline()
render()
}
Log.d(TAG, "Image converted to height map")
} catch (e: Exception) {
Log.e(TAG, "Failed to process selected image", e)
}
}
How it works:
- Load and decode the image through ContentResolver.
- Resize it to 64 × 64 with createScaledBitmap.
- Extract the color channels. Android stores each pixel as a 32-bit ARGB integer, so bit shifts pull out red, green, and blue.
- Calculate brightness with the BT.601 formula. Human eyes are more sensitive to green than to blue, so green carries the largest weight. Dividing by 255 normalizes the result to the 0–1 range.
- Store the height map and rebuild the pipeline so the new values reach the GPU.
After the loop, black pixels become 0.0, white pixels become 1.0, and the photo is now a grid of numbers.
Step 4: Write the Shaders and Build the Render Pipeline
This step is the heart of the app. Add createPipeline() to WebGpuRenderer. It writes the WGSL shader code as a Kotlin string, compiles it, and creates a render pipeline: the recipe that tells the GPU which shaders to run and how to assemble triangles.
private fun createPipeline() {
val gpu = webGpu ?: return
val heights = heightMap.joinToString(",") {
"%.4f".format(java.util.Locale.US, it)
}
val shaderCode = """
const HEIGHTS: array<f32, ${GRID_SIZE * GRID_SIZE}> =
array<f32, ${GRID_SIZE * GRID_SIZE}>(
$heights
);
struct VertexOutput {
@builtin(position) position: vec4f,
@location(0) height: f32,
};
@vertex
fn vs_main(
@builtin(vertex_index) vertexIndex: u32
) -> VertexOutput {
let grid = ${GRID_SIZE}u;
let cells = grid - 1u;
let cellIndex = vertexIndex / 6u;
let corner = vertexIndex % 6u;
let cellX = cellIndex % cells;
let cellY = cellIndex / cells;
var ix = cellX;
var iy = cellY;
// Two triangles per grid cell.
switch corner {
case 0u: { ix = cellX; iy = cellY; }
case 1u: { ix = cellX + 1u; iy = cellY; }
case 2u: { ix = cellX; iy = cellY + 1u; }
case 3u: { ix = cellX; iy = cellY + 1u; }
case 4u: { ix = cellX + 1u; iy = cellY; }
default: { ix = cellX + 1u; iy = cellY + 1u; }
}
let x = f32(ix) / f32(cells);
let y = f32(iy) / f32(cells);
let heightIndex = iy * grid + ix;
let h = HEIGHTS[heightIndex] * 0.65;
// Isometric-style relief projection.
let isoX = (x - y) * 0.75;
let isoY = (x + y) * 0.35 - h * 0.8;
var out: VertexOutput;
out.position = vec4f(isoX, -isoY + 0.30, 0.5, 1.0);
out.height = HEIGHTS[heightIndex];
return out;
}
@fragment
fn fs_main(input: VertexOutput) -> @location(0) vec4f {
let h = clamp(input.height, 0.0, 1.0);
let lowColor = vec3f(0.05, 0.10, 0.20);
let midColor = vec3f(0.10, 0.55, 0.75);
let highColor = vec3f(0.90, 0.95, 1.00);
var color = mix(lowColor, midColor, h * 2.0);
if (h > 0.5) {
color = mix(midColor, highColor, (h - 0.5) * 2.0);
}
return vec4f(color, 1.0);
}
""".trimIndent()
val shaderModule = gpu.device.createShaderModule(
GPUShaderModuleDescriptor(
shaderSourceWGSL = GPUShaderSourceWGSL(shaderCode)
)
)
renderPipeline = gpu.device.createRenderPipeline(
GPURenderPipelineDescriptor(
vertex = GPUVertexState(
module = shaderModule,
entryPoint = "vs_main"
),
fragment = GPUFragmentState(
module = shaderModule,
entryPoint = "fs_main",
targets = arrayOf(
GPUColorTargetState(TextureFormat.RGBA8Unorm)
)
),
primitive = GPUPrimitiveState(
topology = PrimitiveTopology.TriangleList,
frontFace = FrontFace.CCW,
cullMode = CullMode.None
)
)
)
}
Let’s walk through the shader code piece by piece.
4.1 Pass the height map to the shader
The 4,096 height values are written straight into the shader source as a constant array named HEIGHTS. This keeps the example simple: no GPU buffers are needed, and the shader is rebuilt whenever a new image is selected.
4.2 Generate the terrain in the vertex shader
Notice that the app never builds a mesh in Kotlin. The GPU is simply asked to draw 23,814 vertices, and vs_main works out where each one belongs from its index alone:
- vertexIndex / 6u gives the grid square the vertex belongs to.
- vertexIndex % 6u gives which of the square’s six corners it is.
- The switch maps those six corners onto two triangles.
For example, vertex 7 belongs to square 1 (7 ÷ 6) and is corner 1 of that square (7 mod 6).
The vertex then looks up its height and applies the core rule of the app, in one line:
let h = HEIGHTS[heightIndex] * 0.65;
The factor 0.65 controls how tall the hills become. Finally, an isometric projection tilts the flat grid so you look down at it from an angle, while h lifts each point upward:
let isoX = (x - y) * 0.75;
let isoY = (x + y) * 0.35 - h * 0.8;
4.3 Color the surface in the fragment shader
fs_main colors each pixel by height, using three colors: navy for valleys, teal for slopes, and near-white for peaks. The lower half of the range blends navy into teal, and the upper half blends teal into white.
Why * 2.0? Each blend covers only half of the 0–1 range. Multiplying by two stretches each half to a full 0–1 blend, so both halves meet smoothly at h = 0.5 with no visible stripe.
4.4 Build the pipeline
The last part of the function compiles the WGSL into a shader module and connects the two entry points, vs_main and fs_main. PrimitiveTopology.TriangleList tells the GPU to treat every three vertices as one triangle.
Step 5: Draw a Frame and Release Resources
Add the last two functions to WebGpuRenderer. render() records the drawing commands, submits them to the GPU, and shows the result on screen:
fun render() {
val gpu = webGpu ?: return
val pipeline = renderPipeline ?: return
try {
val surfaceTexture = gpu.webgpuSurface.getCurrentTexture()
val commandEncoder = gpu.device.createCommandEncoder()
val renderPass = commandEncoder.beginRenderPass(
GPURenderPassDescriptor(
colorAttachments = arrayOf(
GPURenderPassColorAttachment(
GPUColor(0.025, 0.035, 0.07, 1.0),
surfaceTexture.texture.createView(),
loadOp = Clear,
storeOp = Store
)
)
)
)
renderPass.setPipeline(pipeline)
renderPass.draw(VERTEX_COUNT)
renderPass.end()
gpu.device.queue.submit(arrayOf(commandEncoder.finish()))
gpu.webgpuSurface.present()
} catch (e: Exception) {
Log.e(TAG, "WebGPU render failed", e)
}
}
fun cleanup() {
webGpu?.close()
webGpu = null
renderPipeline = null
Log.d(TAG, "WebGPU resources cleaned up")
}
}
A frame is drawn in four moves:
- Get the current texture from the surface; this is the image you draw into.
- Begin a render pass that clears the background to a dark navy color.
- Draw VERTEX_COUNT vertices with the pipeline from Step 4.
- Submit and present the commands so the frame appears on screen.
cleanup() releases the GPU resources when the surface is destroyed. The closing brace ends the WebGpuRenderer class, which is now complete.
Step 6: Host the Renderer in Compose
Now connect the renderer to the UI. Open WebGpuSurface.kt and add this composable:
private const val TAG = "WebGpuSurface"
@Composable
fun WebGpuSurface(
modifier: Modifier = Modifier,
imageUri: Uri?,
generateRequested: Boolean,
onGenerationHandled: () -> Unit
) {
val context = LocalContext.current
val renderer = remember(context) {
WebGpuRenderer(context.applicationContext)
}
// Keep the renderer alive while the external surface exists.
AndroidExternalSurface(modifier = modifier.fillMaxSize()) {
onSurface { surface, width, height ->
try {
withContext(Dispatchers.Default) {
renderer.init(surface = surface, width = width, height = height)
Log.d(TAG, "Surface initialized: ${width}x${height}")
renderer.render()
// Keep this coroutine alive until Android destroys the surface.
awaitCancellation()
}
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e(TAG, "WebGPU surface failed", e)
} finally {
withContext(NonCancellable + Dispatchers.Default) {
renderer.cleanup()
}
}
}
}
// Process the image when Generate 3D is pressed.
LaunchedEffect(imageUri, generateRequested) {
if (!generateRequested) return@LaunchedEffect
try {
if (imageUri == null) {
Log.w(TAG, "Generate requested without an image")
return@LaunchedEffect
}
Log.d(TAG, "Generating relief from image: $imageUri")
withContext(Dispatchers.Default) {
renderer.setImageUri(imageUri)
}
Log.d(TAG, "Image processing request completed")
} catch (e: CancellationException) {
throw e
} catch (e: Exception) {
Log.e(TAG, "Generate 3D failed", e)
} finally {
onGenerationHandled()
}
}
}
This composable has two jobs:
- Manage the surface. AndroidExternalSurface provides a real Android Surface. When it becomes available, the renderer initializes WebGPU and draws the first frame (a flat plane). awaitCancellation() keeps the coroutine alive while the surface exists, and the finally block calls cleanup() when it is destroyed. NonCancellable ensures the cleanup still runs even though the coroutine is being cancelled.
- React to the Generate request. When generateRequested becomes true, the LaunchedEffect passes the image to the renderer on a background dispatcher. The finally block always calls onGenerationHandled(), so the button never stays stuck.
Compose knows nothing about shaders, and the renderer knows nothing about buttons. This clean boundary keeps both sides easy to read.
Step 7: Build the Main Screen
Finally, replace the contents of MainActivity.kt with the screen that ties everything together:
class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContent {
MaterialTheme {
Surface(
modifier = Modifier.fillMaxSize(),
color = MaterialTheme.colorScheme.background
) {
ImageTo3DScreen()
}
}
}
}
}
@Composable
private fun ImageTo3DScreen() {
var selectedImageUri by remember { mutableStateOf<Uri?>(null) }
var generateRequested by remember { mutableStateOf(false) }
val imagePicker = rememberLauncherForActivityResult(
contract = ActivityResultContracts.GetContent()
) { uri ->
selectedImageUri = uri
generateRequested = false
}
Column(
modifier = Modifier
.fillMaxSize()
.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Text(
text = "2D Image to 3D",
style = MaterialTheme.typography.headlineMedium
)
Text(
text = "Choose an image and turn it into a 3D relief.",
style = MaterialTheme.typography.bodyMedium
)
OutlinedButton(
onClick = { imagePicker.launch("image/*") },
modifier = Modifier.fillMaxWidth()
) {
Text("Upload Image")
}
val uri = selectedImageUri
if (uri != null) {
Image(
painter = rememberAsyncImagePainter(uri),
contentDescription = "Selected image preview",
modifier = Modifier
.fillMaxWidth()
.height(200.dp)
.clip(RoundedCornerShape(12.dp)),
contentScale = ContentScale.Fit
)
Text(
text = "Image selected successfully",
style = MaterialTheme.typography.bodySmall
)
Button(
onClick = { generateRequested = true },
modifier = Modifier.fillMaxWidth()
) {
Text("Generate 3D")
}
} else {
Box(
modifier = Modifier
.fillMaxWidth()
.height(200.dp)
.clip(RoundedCornerShape(12.dp))
.background(Color(0xFF202124)),
contentAlignment = Alignment.Center
) {
Text(
text = "Upload an image to get started",
color = Color.White
)
}
}
Spacer(modifier = Modifier.height(4.dp))
Text(
text = "3D Preview",
style = MaterialTheme.typography.titleMedium
)
Box(
modifier = Modifier
.fillMaxWidth()
.weight(1f)
.clip(RoundedCornerShape(12.dp))
.background(Color(0xFF111318))
) {
WebGpuSurface(
modifier = Modifier.fillMaxSize(),
imageUri = selectedImageUri,
generateRequested = generateRequested,
onGenerationHandled = { generateRequested = false }
)
}
}
}
The screen holds two pieces of state:
- selectedImageUri stores the image chosen through the system picker.
- generateRequested becomes true when the user taps Generate 3D, and WebGpuSurface resets it to false once the image is processed.
The layout shows an upload button, an image preview (or a placeholder before an image is chosen), the Generate 3D button, and the 3D preview area, which takes the remaining screen space.
Step 8: Run the App
- Connect your Android device and click Run in Android Studio.
- When the app opens, the 3D preview shows a flat plane. This confirms WebGPU is initialized.
- Tap Upload Image and choose a photo.
- Tap Generate 3D. The photo appears as a relief: bright areas rise as white peaks, and dark areas settle into navy valleys.
If the preview stays black, open Logcat and filter by WebGpuRenderer or WebGpuSurface. The log messages added throughout the code show exactly which stage failed.
Conclusion
In eight steps, you built an Android app that turns a photo into a 3D relief. Kotlin converts pixel brightness into a height map, the vertex shader generates the terrain from vertex indices alone, the fragment shader colors it by height, and Compose hosts it all through a clean, simple UI. It is a compact example of how WebGPU and Jetpack Compose can work together, and a solid starting point for exploring GPU rendering on Android.
For full sample codes are available here.
Exploring WebGPU on Android: Turning a 2D Image into a 3D Relief with Jetpack Compose was originally published in ProAndroidDev on Medium, where people are continuing the conversation by highlighting and responding to this story.