`evorl.ec.optimizers`¶

Package Contents¶

Classes¶

`ARS`	Augmented Random Search.
`ERLGA`
`ERLGAMod`
`EvoOptimizer`	By default, all EvoOptimizer maximize the fitness.
`ExponentialScheduleSpec`	Specification for an exponential schedule for HyperParam.
`OpenES`	OpenAI ES.
`OpenESNoiseTable`	OpenAI ES with noise table.
`SepCEM`	Sep Cross-Entropy Method.
`VanillaES`	Canonical Evolution Strategies.
`VanillaESMod`	Variant of VanillaES.
`VanillaGA`	Vanilla Genetic Algorithm.

Data¶

ECState

API¶

class evorl.ec.optimizers.ARS[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

Augmented Random Search.

Paper: Simple random search of static linear policies is competitive for reinforcement learning

ask(state: evorl.ec.optimizers.ars.ARSState) → tuple[evorl.types.Params, evorl.ec.optimizers.ec_optimizer.ECState][source]¶

fitness_std_eps: float¶: 1e-08

init(mean: evorl.types.Params, key: chex.PRNGKey) → evorl.ec.optimizers.ars.ARSState[source]¶

lr: float¶: None

noise_std: float¶: None

num_elites: int¶: None

optimizer: optax.GradientTransformation¶: ‘pytree_field(…)’

optimizer_name: str¶: ‘sgd’

pop_size: int¶: None

tell(state: evorl.ec.optimizers.ars.ARSState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.ars.ARSState][source]¶

evorl.ec.optimizers.ECState¶: None

class evorl.ec.optimizers.ERLGA[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

ask(state: evorl.ec.optimizers.erl_ga.ERLGAState) → tuple[chex.ArrayTree, evorl.ec.optimizers.erl_ga.ERLGAState][source]¶

enable_crossover: bool¶: True

init(pop, key) → evorl.ec.optimizers.erl_ga.ERLGAState[source]¶

mut_strength: float¶: 0.1

num_crossover_frac: float¶: 0.1

num_elites: int¶: None

num_mutation_frac: float¶: 0.1

pop_size: int¶: None

reset_prob: float¶: 0.1

super_mut_prob: float¶: 0.05

super_mut_strength: float¶: 10.0

tell(state: evorl.ec.optimizers.erl_ga.ERLGAState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.erl_ga.ERLGAState][source]¶

tournament_size: int¶: 3

vec_relative_prob: float¶: 0.0

weight_max_magnitude: float¶: 1000000.0

class evorl.ec.optimizers.ERLGAMod[source]¶

Bases: evorl.ec.optimizers.erl_ga.ERLGA

external_size: int¶: None

init(pop, key) → evorl.ec.optimizers.erl_ga.ERLGAModState[source]¶

tell_external(state: evorl.ec.optimizers.erl_ga.ERLGAModState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.erl_ga.ERLGAModState][source]¶

class evorl.ec.optimizers.EvoOptimizer[source]¶

Bases: evorl.types.PyTreeNode

By default, all EvoOptimizer maximize the fitness.

This is different from the behavior in EvoX.

abstract ask(state: evorl.ec.optimizers.ec_optimizer.ECState) → tuple[chex.ArrayTree, evorl.ec.optimizers.ec_optimizer.ECState][source]¶: Generate new candidate solutions.

abstract init(*args, **kwargs) → evorl.ec.optimizers.ec_optimizer.ECState[source]¶

abstract tell(state: evorl.ec.optimizers.ec_optimizer.ECState, fitnesses: chex.ArrayTree) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.ec_optimizer.ECState][source]¶

Update the optimizer state based on the fitnesses of the candidate solutions.

Parameters:

state – The current optimizer state
fitnesses – The fitnesses of the candidate solutions

class evorl.ec.optimizers.ExponentialScheduleSpec[source]¶

Bases: evorl.types.PyTreeData

Specification for an exponential schedule for HyperParam.

decay: float¶: None

final: float¶: None

init: float¶: None

class evorl.ec.optimizers.OpenES[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

OpenAI ES.

ask(state: evorl.ec.optimizers.ec_optimizer.ECState) → tuple[chex.ArrayTree, evorl.ec.optimizers.ec_optimizer.ECState][source]¶: Generate new candidate solutions.

fitness_shaping_fn: collections.abc.Callable[[chex.Array], chex.Array]¶: ‘pytree_field(…)’

init(mean: evorl.types.Params, key: chex.PRNGKey) → evorl.ec.optimizers.ec_optimizer.ECState[source]¶

lr_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

mirror_sampling: bool¶: True

noise_std_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

optimizer: optax.GradientTransformation¶: ‘pytree_field(…)’

optimizer_name: str¶: ‘adam’

pop_size: int¶: None

tell(state: evorl.ec.optimizers.ec_optimizer.ECState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.openes.OpenESState][source]¶: Update the optimizer state based on the fitnesses of the candidate solutions.

weight_decay: float | None¶: None

class evorl.ec.optimizers.OpenESNoiseTable[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

OpenAI ES with noise table.

ask(state: evorl.ec.optimizers.ec_optimizer.ECState) → tuple[chex.ArrayTree, evorl.ec.optimizers.ec_optimizer.ECState][source]¶: Generate new candidate solutions.

fitness_shaping_fn: collections.abc.Callable[[chex.Array], chex.Array]¶: ‘pytree_field(…)’

init(mean: evorl.types.Params, key: chex.PRNGKey) → evorl.ec.optimizers.ec_optimizer.ECState[source]¶

lr_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

mirror_sampling: bool¶: True

noise_std_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

noise_table_size: int¶: None

optimizer: optax.GradientTransformation¶: ‘pytree_field(…)’

optimizer_name: str¶: ‘adam’

pop_size: int¶: None

tell(state: evorl.ec.optimizers.ec_optimizer.ECState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.openes.OpenESState][source]¶: Update the optimizer state based on the fitnesses of the candidate solutions.

weight_decay: float | None¶: None

class evorl.ec.optimizers.SepCEM[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

Sep Cross-Entropy Method.

ask(state: evorl.ec.optimizers.cem.SepCEMState) → tuple[chex.ArrayTree, evorl.ec.optimizers.ec_optimizer.ECState][source]¶

cov_eps_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

elite_weights: chex.Array¶: ‘pytree_field(…)’

init(mean: evorl.types.Params, key: chex.PRNGKey) → evorl.ec.optimizers.cem.SepCEMState[source]¶

mirror_sampling: bool¶: False

num_elites: int¶: None

pop_size: int¶: None

rank_weight_shift: float¶: 1.0

tell(state: evorl.ec.optimizers.cem.SepCEMState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.cem.SepCEMState][source]¶

weighted_update: bool¶: True

class evorl.ec.optimizers.VanillaES[source]¶

Bases: evorl.ec.optimizers.ec_optimizer.EvoOptimizer

Canonical Evolution Strategies.

Paper: Back to basics: Benchmarking canonical evolution strategies for playing atari

ask(state: evorl.ec.optimizers.vanilla_es.VanillaESState) → tuple[evorl.types.Params, evorl.ec.optimizers.vanilla_es.VanillaESState][source]¶

elite_weights: chex.Array¶: ‘pytree_field(…)’

init(mean: evorl.types.Params, key: chex.PRNGKey) → evorl.ec.optimizers.vanilla_es.VanillaESState[source]¶

noise_std_schedule: evorl.ec.optimizers.utils.ExponentialScheduleSpec¶: None

num_elites: int¶: None

pop_size: int¶: None

tell(state: evorl.ec.optimizers.vanilla_es.VanillaESState, fitnesses: chex.Array) → tuple[evorl.types.PyTreeDict, evorl.ec.optimizers.vanilla_es.VanillaESState][source]¶

class evorl.ec.optimizers.VanillaESMod[source]¶

Bases: evorl.ec.optimizers.vanilla_es.VanillaES

Variant of VanillaES.

Add external_size number of external individuals and corresponding fitnesses during the ES update by tell_external()

Variables:

external_size – number of external individuals
mix_strategy –
strategy to mix external individuals with the elites.
- ”always”: always mix external individuals with elites
- ”normal”: concat external individuals to the population and select num_elites elites from the combined population.