evorl.replay_buffers.lap_replay_buffer

LAP (Loss-Adjusted Prioritization) Replay Buffer.

Module Contents

Classes

LAPReplayBuffer

ReplayBuffer with Loss-Adjusted Prioritization from TD7 paper.

API

class evorl.replay_buffers.lap_replay_buffer.LAPReplayBuffer[source]

Bases: evorl.replay_buffers.prioritized_replay_buffer.PrioritizedReplayBuffer

ReplayBuffer with Loss-Adjusted Prioritization from TD7 paper.

LAP is a variation of Prioritized Experience Replay (PER) that uses proportional sampling but explicitly drops Importance Sampling (IS) weights, setting them uniformly to 1.0. It leverages the parent PrioritizedReplayBuffer class for alpha-scaled priority sampling.

Variables:

  • capacity – the maximum capacity of the replay buffer.

  • sample_batch_size – the batch size for sample().

  • min_sample_timesteps – the minimum number of timesteps before sampling.

  • alpha – the exponent determining how much prioritization is used (0 = uniform, 1 = full).

sample(buffer_state: evorl.replay_buffers.prioritized_replay_buffer.PrioritizedReplayBufferState, key: chex.PRNGKey, beta: float | chex.Array = 0.0) → tuple[chex.ArrayTree, chex.Array, evorl.replay_buffers.prioritized_replay_buffer.PrioritizedReplayBufferState][source]

Sample a batch proportional to alpha-scaled priorities without IS weights.

Parameters:

  • buffer_state – Current buffer state.

  • key – PRNG key.

  • beta – Ignored mapping variable to match parent interface.

Returns:

A tuple of (batch, weights, updated_buffer_state). The IS weights are uniformly set to 1.0.