JAX

What is JAX

JAX is a high-performance numerical computing library introduced by Google, providing a NumPy-like API and supporting features such as GPU/TPU acceleration, automatic differentiation, just-in-time compilation (JIT), and vectorization. JAX optimizes code through the XLA (Accelerated Linear Algebra) compiler, significantly improving runtime efficiency and excelling in large-scale data processing and machine learning. JAX supports automatic differentiation, making it easy to compute function gradients, and is suitable for optimization algorithms. JAX's asynchronous execution model and immutable array design make it superior to traditionalNumPy, and it is an important tool in modern scientific computing and machine learning research.

Main features of JAX

• Automatic differentiation: Throughjax.gradand other functions, it automatically computes function gradients, supports higher-order derivatives, and is widely used in model training in machine learning.
• Just-in-time compilation (JIT): Usejax.jitto compile Python functions into optimized machine code, significantly improving runtime efficiency, with notable results in large-scale computation.
• Vectorization: Throughjax.vmap, it automatically vectorizes functions, avoiding manual loops and improving code efficiency and readability.
• Parallelization: Usejax.pmapto support parallel computation across multiple devices (such as GPU and TPU), accelerating large-scale task processing.
• Hardware acceleration: Supports running code on CPU, GPU, and TPU, fully utilizing the hardware's parallel computing capabilities.
• Program transformation: Provides a rich set of program transformation tools, such asjax.lax, used to build more complex program logic and improve code flexibility and extensibility.

How to use JAX

• Environment setup and installation：
  • Create a Python environment: Use conda to create a dedicated Python environment.

conda create -n jax_test python=3.13 -y
conda activate jax_test

• • Install the JAX library: Choose the appropriate JAX version according to the hardware configuration.

pip install jupyter numpy "jax[cuda12]" matplotlib pillow

• Use the main features of JAX：
  • Automatic differentiation: Usejax.gradto automatically compute function gradients.

import jax
import jax.numpy as jnp

def cubic_sum(x):
    return jnp.sum(x**3)

grad_cubic_sum = jax.grad(cubic_sum)
x_input = jnp.arange(1.0, 5.0)
gradient = grad_cubic_sum(x_input)
print("梯度 df/dx:", gradient)

• • Just-in-time compilation (JIT): Usejax.jitto compile functions into optimized machine code.

@jax.jit
def selu_jax_jit(x):
    return 1.0507 * jnp.where(x > 0, x, 1.67326 * jnp.exp(x) - 1.67326)

x_jax = jnp.random.normal(jax.random.PRNGKey(0), (10000, 10000))
result_jax_jit = selu_jax_jit(x_jax)

• • Vectorization: Usejax.vmapto automatically vectorize functions.

def mat_vec_product(matrix, vector):
    return jnp.dot(matrix, vector)

batched_mat_vec = jax.vmap(mat_vec_product, in_axes=(None, 0))
matrix_jax = jnp.random.normal(jax.random.PRNGKey(0), (10000, 10000))
vectors_jax = jnp.random.normal(jax.random.PRNGKey(1), (128, 10000))
result_vmap = batched_mat_vec(matrix_jax, vectors_jax)