# {py:mod}`evorl.algorithms.impala`

```{py:module} evorl.algorithms.impala
```

```{autodoc2-docstring} evorl.algorithms.impala
:parser: autodoc2_docstrings_parser
:allowtitles:
```

## Module Contents

### Classes

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`IMPALAAgent <evorl.algorithms.impala.IMPALAAgent>`
  -
* - {py:obj}`IMPALANetworkParams <evorl.algorithms.impala.IMPALANetworkParams>`
  - ```{autodoc2-docstring} evorl.algorithms.impala.IMPALANetworkParams
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`IMPALAWorkflow <evorl.algorithms.impala.IMPALAWorkflow>`
  - ```{autodoc2-docstring} evorl.algorithms.impala.IMPALAWorkflow
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### Functions

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`compute_pg_advantage <evorl.algorithms.impala.compute_pg_advantage>`
  - ```{autodoc2-docstring} evorl.algorithms.impala.compute_pg_advantage
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`compute_vtrace <evorl.algorithms.impala.compute_vtrace>`
  - ```{autodoc2-docstring} evorl.algorithms.impala.compute_vtrace
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`make_mlp_impala_agent <evorl.algorithms.impala.make_mlp_impala_agent>`
  - ```{autodoc2-docstring} evorl.algorithms.impala.make_mlp_impala_agent
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### API

`````{py:class} IMPALAAgent
:canonical: evorl.algorithms.impala.IMPALAAgent

Bases: {py:obj}`evorl.agent.Agent`

````{py:attribute} adv_mode
:canonical: evorl.algorithms.impala.IMPALAAgent.adv_mode
:type: str
:value: >
   'pytree_field(...)'

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.adv_mode
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} clip_c_threshold
:canonical: evorl.algorithms.impala.IMPALAAgent.clip_c_threshold
:type: float
:value: >
   1.0

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.clip_c_threshold
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} clip_pg_rho_threshold
:canonical: evorl.algorithms.impala.IMPALAAgent.clip_pg_rho_threshold
:type: float
:value: >
   1.0

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.clip_pg_rho_threshold
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} clip_rho_threshold
:canonical: evorl.algorithms.impala.IMPALAAgent.clip_rho_threshold
:type: float
:value: >
   1.0

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.clip_rho_threshold
:parser: autodoc2_docstrings_parser
```

````

````{py:method} compute_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.impala.IMPALAAgent.compute_actions

````

````{py:attribute} continuous_action
:canonical: evorl.algorithms.impala.IMPALAAgent.continuous_action
:type: bool
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.continuous_action
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} discount
:canonical: evorl.algorithms.impala.IMPALAAgent.discount
:type: float
:value: >
   0.99

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.discount
:parser: autodoc2_docstrings_parser
```

````

````{py:method} evaluate_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.impala.IMPALAAgent.evaluate_actions

````

````{py:method} init(obs_space: evorl.envs.Space, action_space: evorl.envs.Space, key: chex.PRNGKey) -> evorl.agent.AgentState
:canonical: evorl.algorithms.impala.IMPALAAgent.init

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.init
:parser: autodoc2_docstrings_parser
```

````

````{py:method} loss(agent_state: evorl.agent.AgentState, trajectory: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> evorl.types.LossDict
:canonical: evorl.algorithms.impala.IMPALAAgent.loss

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.loss
:parser: autodoc2_docstrings_parser
```

````

````{py:property} normalize_obs
:canonical: evorl.algorithms.impala.IMPALAAgent.normalize_obs

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.normalize_obs
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} obs_preprocessor
:canonical: evorl.algorithms.impala.IMPALAAgent.obs_preprocessor
:type: typing.Any
:value: >
   'pytree_field(...)'

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.obs_preprocessor
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} policy_network
:canonical: evorl.algorithms.impala.IMPALAAgent.policy_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.policy_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} value_network
:canonical: evorl.algorithms.impala.IMPALAAgent.value_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.value_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} vtrace_lambda
:canonical: evorl.algorithms.impala.IMPALAAgent.vtrace_lambda
:type: float
:value: >
   1.0

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAAgent.vtrace_lambda
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} IMPALANetworkParams
:canonical: evorl.algorithms.impala.IMPALANetworkParams

Bases: {py:obj}`evorl.types.PyTreeData`

```{autodoc2-docstring} evorl.algorithms.impala.IMPALANetworkParams
:parser: autodoc2_docstrings_parser
```

````{py:attribute} policy_params
:canonical: evorl.algorithms.impala.IMPALANetworkParams.policy_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.impala.IMPALANetworkParams.policy_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} value_params
:canonical: evorl.algorithms.impala.IMPALANetworkParams.value_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.impala.IMPALANetworkParams.value_params
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} IMPALAWorkflow(env: evorl.envs.Env, agent: evorl.agent.Agent, optimizer: optax.GradientTransformation, evaluator: evorl.evaluators.Evaluator, config: omegaconf.DictConfig)
:canonical: evorl.algorithms.impala.IMPALAWorkflow

Bases: {py:obj}`evorl.workflows.OnPolicyWorkflow`

```{autodoc2-docstring} evorl.algorithms.impala.IMPALAWorkflow
:parser: autodoc2_docstrings_parser
```

````{py:method} learn(state: evorl.types.State) -> evorl.types.State
:canonical: evorl.algorithms.impala.IMPALAWorkflow.learn

````

````{py:method} name()
:canonical: evorl.algorithms.impala.IMPALAWorkflow.name
:classmethod:

````

````{py:method} step(state: evorl.types.State) -> tuple[evorl.metrics.MetricBase, evorl.types.State]
:canonical: evorl.algorithms.impala.IMPALAWorkflow.step

````

`````

````{py:function} compute_pg_advantage(vtrace, v_t, v_t_plus_1, rewards, terminations, discount=0.99, lambda_=1.0, mode='official')
:canonical: evorl.algorithms.impala.compute_pg_advantage

```{autodoc2-docstring} evorl.algorithms.impala.compute_pg_advantage
:parser: autodoc2_docstrings_parser
```
````

````{py:function} compute_vtrace(rho_t, v_t, v_t_plus_1, rewards, dones, terminations, discount=0.99, lambda_=1.0, clip_rho_threshold=1.0, clip_c_threshold=1.0)
:canonical: evorl.algorithms.impala.compute_vtrace

```{autodoc2-docstring} evorl.algorithms.impala.compute_vtrace
:parser: autodoc2_docstrings_parser
```
````

````{py:function} make_mlp_impala_agent(action_space: evorl.envs.Space, discount: float = 0.99, vtrace_lambda: float = 1.0, clip_rho_threshold: float = 1.0, clip_c_threshold: float = 1.0, clip_pg_rho_threshold: float = 1.0, adv_mode: str = 'official', actor_hidden_layer_sizes: tuple[int] = (256, 256), critic_hidden_layer_sizes: tuple[int] = (256, 256), normalize_obs: bool = False, policy_obs_key: str = '', value_obs_key: str = '')
:canonical: evorl.algorithms.impala.make_mlp_impala_agent

```{autodoc2-docstring} evorl.algorithms.impala.make_mlp_impala_agent
:parser: autodoc2_docstrings_parser
```
````