# {py:mod}`evorl.algorithms.td7`

```{py:module} evorl.algorithms.td7
```

```{autodoc2-docstring} evorl.algorithms.td7
:parser: autodoc2_docstrings_parser
:allowtitles:
```

## Module Contents

### Classes

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`TD7Actor <evorl.algorithms.td7.TD7Actor>`
  -
* - {py:obj}`TD7Agent <evorl.algorithms.td7.TD7Agent>`
  - ```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`TD7Critic <evorl.algorithms.td7.TD7Critic>`
  -
* - {py:obj}`TD7Encoder <evorl.algorithms.td7.TD7Encoder>`
  -
* - {py:obj}`TD7NetworkParams <evorl.algorithms.td7.TD7NetworkParams>`
  -
* - {py:obj}`TD7TrainMetric <evorl.algorithms.td7.TD7TrainMetric>`
  -
* - {py:obj}`TD7Workflow <evorl.algorithms.td7.TD7Workflow>`
  -
````

### Functions

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`avg_l1_norm <evorl.algorithms.td7.avg_l1_norm>`
  - ```{autodoc2-docstring} evorl.algorithms.td7.avg_l1_norm
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`make_td7_agent <evorl.algorithms.td7.make_td7_agent>`
  - ```{autodoc2-docstring} evorl.algorithms.td7.make_td7_agent
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### API

`````{py:class} TD7Actor
:canonical: evorl.algorithms.td7.TD7Actor

Bases: {py:obj}`flax.linen.Module`

````{py:attribute} action_size
:canonical: evorl.algorithms.td7.TD7Actor.action_size
:type: int
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7Actor.action_size
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} hidden_layer_sizes
:canonical: evorl.algorithms.td7.TD7Actor.hidden_layer_sizes
:type: typing.Sequence[int]
:value: >
   (256, 256)

```{autodoc2-docstring} evorl.algorithms.td7.TD7Actor.hidden_layer_sizes
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} state_emb_dim
:canonical: evorl.algorithms.td7.TD7Actor.state_emb_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Actor.state_emb_dim
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} z_s_dim
:canonical: evorl.algorithms.td7.TD7Actor.z_s_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Actor.z_s_dim
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7Agent
:canonical: evorl.algorithms.td7.TD7Agent

Bases: {py:obj}`evorl.agent.Agent`

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent
:parser: autodoc2_docstrings_parser
```

````{py:method} actor_loss(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> evorl.types.LossDict
:canonical: evorl.algorithms.td7.TD7Agent.actor_loss

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.actor_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} actor_network
:canonical: evorl.algorithms.td7.TD7Agent.actor_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.actor_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} clip_policy_noise
:canonical: evorl.algorithms.td7.TD7Agent.clip_policy_noise
:type: float
:value: >
   0.5

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.clip_policy_noise
:parser: autodoc2_docstrings_parser
```

````

````{py:method} compute_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.td7.TD7Agent.compute_actions

````

````{py:method} critic_loss(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> evorl.types.LossDict
:canonical: evorl.algorithms.td7.TD7Agent.critic_loss

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.critic_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic_network
:canonical: evorl.algorithms.td7.TD7Agent.critic_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.critic_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} discount
:canonical: evorl.algorithms.td7.TD7Agent.discount
:type: float
:value: >
   0.99

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.discount
:parser: autodoc2_docstrings_parser
```

````

````{py:method} encoder_loss(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> evorl.types.LossDict
:canonical: evorl.algorithms.td7.TD7Agent.encoder_loss

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.encoder_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} encoder_network
:canonical: evorl.algorithms.td7.TD7Agent.encoder_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.encoder_network
:parser: autodoc2_docstrings_parser
```

````

````{py:method} evaluate_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.td7.TD7Agent.evaluate_actions

````

````{py:attribute} exploration_epsilon
:canonical: evorl.algorithms.td7.TD7Agent.exploration_epsilon
:type: float
:value: >
   0.1

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.exploration_epsilon
:parser: autodoc2_docstrings_parser
```

````

````{py:method} init(obs_space: evorl.envs.Space, action_space: evorl.envs.Space, key: chex.PRNGKey) -> evorl.agent.AgentState
:canonical: evorl.algorithms.td7.TD7Agent.init

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.init
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} min_priority
:canonical: evorl.algorithms.td7.TD7Agent.min_priority
:type: float
:value: >
   1.0

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.min_priority
:parser: autodoc2_docstrings_parser
```

````

````{py:property} normalize_obs
:canonical: evorl.algorithms.td7.TD7Agent.normalize_obs

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.normalize_obs
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} obs_preprocessor
:canonical: evorl.algorithms.td7.TD7Agent.obs_preprocessor
:type: typing.Any
:value: >
   'pytree_field(...)'

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.obs_preprocessor
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} policy_noise
:canonical: evorl.algorithms.td7.TD7Agent.policy_noise
:type: float
:value: >
   0.2

```{autodoc2-docstring} evorl.algorithms.td7.TD7Agent.policy_noise
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7Critic
:canonical: evorl.algorithms.td7.TD7Critic

Bases: {py:obj}`flax.linen.Module`

````{py:attribute} hidden_layer_sizes
:canonical: evorl.algorithms.td7.TD7Critic.hidden_layer_sizes
:type: typing.Sequence[int]
:value: >
   (256, 256)

```{autodoc2-docstring} evorl.algorithms.td7.TD7Critic.hidden_layer_sizes
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} state_action_emb_dim
:canonical: evorl.algorithms.td7.TD7Critic.state_action_emb_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Critic.state_action_emb_dim
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} z_s_dim
:canonical: evorl.algorithms.td7.TD7Critic.z_s_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Critic.z_s_dim
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} z_sa_dim
:canonical: evorl.algorithms.td7.TD7Critic.z_sa_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Critic.z_sa_dim
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7Encoder
:canonical: evorl.algorithms.td7.TD7Encoder

Bases: {py:obj}`flax.linen.Module`

````{py:attribute} f_layer_sizes
:canonical: evorl.algorithms.td7.TD7Encoder.f_layer_sizes
:type: typing.Sequence[int]
:value: >
   (256, 256)

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.f_layer_sizes
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} g_layer_sizes
:canonical: evorl.algorithms.td7.TD7Encoder.g_layer_sizes
:type: typing.Sequence[int]
:value: >
   (256, 256)

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.g_layer_sizes
:parser: autodoc2_docstrings_parser
```

````

````{py:method} setup()
:canonical: evorl.algorithms.td7.TD7Encoder.setup

````

````{py:attribute} z_s_dim
:canonical: evorl.algorithms.td7.TD7Encoder.z_s_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.z_s_dim
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} z_sa_dim
:canonical: evorl.algorithms.td7.TD7Encoder.z_sa_dim
:type: int
:value: >
   256

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.z_sa_dim
:parser: autodoc2_docstrings_parser
```

````

````{py:method} zs(obs: jax.Array) -> jax.Array
:canonical: evorl.algorithms.td7.TD7Encoder.zs

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.zs
:parser: autodoc2_docstrings_parser
```

````

````{py:method} zsa(z_s: jax.Array, action: jax.Array) -> jax.Array
:canonical: evorl.algorithms.td7.TD7Encoder.zsa

```{autodoc2-docstring} evorl.algorithms.td7.TD7Encoder.zsa
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7NetworkParams
:canonical: evorl.algorithms.td7.TD7NetworkParams

Bases: {py:obj}`evorl.types.PyTreeData`

````{py:attribute} actor_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.actor_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.actor_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} checkpoint_actor_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.checkpoint_actor_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.checkpoint_actor_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} checkpoint_encoder_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.checkpoint_encoder_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.checkpoint_encoder_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.critic_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.critic_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} encoder_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.encoder_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.encoder_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} fixed_encoder_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.fixed_encoder_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.fixed_encoder_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} fixed_encoder_target_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.fixed_encoder_target_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.fixed_encoder_target_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} target_actor_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.target_actor_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.target_actor_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} target_critic_params
:canonical: evorl.algorithms.td7.TD7NetworkParams.target_critic_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7NetworkParams.target_critic_params
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7TrainMetric
:canonical: evorl.algorithms.td7.TD7TrainMetric

Bases: {py:obj}`evorl.metrics.MetricBase`

````{py:attribute} actor_loss
:canonical: evorl.algorithms.td7.TD7TrainMetric.actor_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7TrainMetric.actor_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic_loss
:canonical: evorl.algorithms.td7.TD7TrainMetric.critic_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7TrainMetric.critic_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} encoder_loss
:canonical: evorl.algorithms.td7.TD7TrainMetric.encoder_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.td7.TD7TrainMetric.encoder_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} raw_loss_dict
:canonical: evorl.algorithms.td7.TD7TrainMetric.raw_loss_dict
:type: evorl.types.LossDict
:value: >
   'metric_field(...)'

```{autodoc2-docstring} evorl.algorithms.td7.TD7TrainMetric.raw_loss_dict
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD7Workflow(env: evorl.envs.Env, agent: evorl.agent.Agent, optimizer: optax.GradientTransformation, evaluator: evorl.evaluators.Evaluator, replay_buffer: evorl.replay_buffers.AbstractReplayBuffer, config: omegaconf.DictConfig)
:canonical: evorl.algorithms.td7.TD7Workflow

Bases: {py:obj}`evorl.algorithms.offpolicy_utils.OffPolicyWorkflowTemplate`

````{py:method} learn(state: evorl.types.State) -> evorl.types.State
:canonical: evorl.algorithms.td7.TD7Workflow.learn

````

````{py:method} name()
:canonical: evorl.algorithms.td7.TD7Workflow.name
:classmethod:

````

````{py:method} step(state: evorl.types.State) -> tuple[evorl.metrics.MetricBase, evorl.types.State]
:canonical: evorl.algorithms.td7.TD7Workflow.step

````

`````

````{py:function} avg_l1_norm(x: jax.Array, eps: float = 1e-08) -> jax.Array
:canonical: evorl.algorithms.td7.avg_l1_norm

```{autodoc2-docstring} evorl.algorithms.td7.avg_l1_norm
:parser: autodoc2_docstrings_parser
```
````

````{py:function} make_td7_agent(action_space: evorl.envs.Space, z_s_dim: int = 256, z_sa_dim: int = 256, f_layer_sizes: typing.Sequence[int] = (256, 256), g_layer_sizes: typing.Sequence[int] = (256, 256), state_emb_dim: int = 256, state_action_emb_dim: int = 256, critic_hidden_layer_sizes: typing.Sequence[int] = (256, 256), actor_hidden_layer_sizes: typing.Sequence[int] = (256, 256), discount: float = 0.99, exploration_epsilon: float = 0.1, policy_noise: float = 0.2, clip_policy_noise: float = 0.5, min_priority: float = 1.0, normalize_obs: bool = False)
:canonical: evorl.algorithms.td7.make_td7_agent

```{autodoc2-docstring} evorl.algorithms.td7.make_td7_agent
:parser: autodoc2_docstrings_parser
```
````