# {py:mod}`evorl.algorithms.contrib.td3_v3`

```{py:module} evorl.algorithms.contrib.td3_v3
```

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3
:parser: autodoc2_docstrings_parser
:allowtitles:
```

## Module Contents

### Classes

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`TD3Agent <evorl.algorithms.contrib.td3_v3.TD3Agent>`
  - ```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`TD3NetworkParams <evorl.algorithms.contrib.td3_v3.TD3NetworkParams>`
  - ```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
* - {py:obj}`TD3TrainMetric <evorl.algorithms.contrib.td3_v3.TD3TrainMetric>`
  -
* - {py:obj}`TD3V3Workflow <evorl.algorithms.contrib.td3_v3.TD3V3Workflow>`
  - ```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3V3Workflow
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### Functions

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`make_mlp_td3_agent <evorl.algorithms.contrib.td3_v3.make_mlp_td3_agent>`
  - ```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.make_mlp_td3_agent
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### Data

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`MISSING_LOSS <evorl.algorithms.contrib.td3_v3.MISSING_LOSS>`
  - ```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.MISSING_LOSS
    :parser: autodoc2_docstrings_parser
    :summary:
    ```
````

### API

````{py:data} MISSING_LOSS
:canonical: evorl.algorithms.contrib.td3_v3.MISSING_LOSS
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.MISSING_LOSS
:parser: autodoc2_docstrings_parser
```

````

`````{py:class} TD3Agent
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent

Bases: {py:obj}`evorl.agent.Agent`

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent
:parser: autodoc2_docstrings_parser
```

````{py:attribute} actor_network
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.actor_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.actor_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} clip_policy_noise
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.clip_policy_noise
:type: float
:value: >
   0.5

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.clip_policy_noise
:parser: autodoc2_docstrings_parser
```

````

````{py:method} compute_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.compute_actions

````

````{py:attribute} critic_network
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.critic_network
:type: flax.linen.Module
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.critic_network
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critics_in_actor_loss
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.critics_in_actor_loss
:type: str
:value: >
   'first'

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.critics_in_actor_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} discount
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.discount
:type: float
:value: >
   0.99

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.discount
:parser: autodoc2_docstrings_parser
```

````

````{py:method} evaluate_actions(agent_state: evorl.agent.AgentState, sample_batch: evorl.sample_batch.SampleBatch, key: chex.PRNGKey) -> tuple[evorl.types.Action, evorl.types.PolicyExtraInfo]
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.evaluate_actions

````

````{py:attribute} exploration_epsilon
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.exploration_epsilon
:type: float
:value: >
   0.5

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.exploration_epsilon
:parser: autodoc2_docstrings_parser
```

````

````{py:method} init(obs_space: evorl.envs.Space, action_space: evorl.envs.Space, key: chex.PRNGKey) -> evorl.agent.AgentState
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.init

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.init
:parser: autodoc2_docstrings_parser
```

````

````{py:property} normalize_obs
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.normalize_obs

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.normalize_obs
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} obs_preprocessor
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.obs_preprocessor
:type: typing.Any
:value: >
   'pytree_field(...)'

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.obs_preprocessor
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} policy_noise
:canonical: evorl.algorithms.contrib.td3_v3.TD3Agent.policy_noise
:type: float
:value: >
   0.2

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3Agent.policy_noise
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD3NetworkParams
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams

Bases: {py:obj}`evorl.types.PyTreeData`

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams
:parser: autodoc2_docstrings_parser
```

````{py:attribute} actor_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.actor_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.actor_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic1_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.critic1_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.critic1_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic2_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.critic2_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.critic2_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} target_actor_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_actor_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_actor_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} target_critic1_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_critic1_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_critic1_params
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} target_critic2_params
:canonical: evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_critic2_params
:type: evorl.types.Params
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3NetworkParams.target_critic2_params
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD3TrainMetric
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric

Bases: {py:obj}`evorl.metrics.MetricBase`

````{py:attribute} actor_loss
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric.actor_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3TrainMetric.actor_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic1_loss
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric.critic1_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3TrainMetric.critic1_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} critic2_loss
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric.critic2_loss
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3TrainMetric.critic2_loss
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} q1
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric.q1
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3TrainMetric.q1
:parser: autodoc2_docstrings_parser
```

````

````{py:attribute} q2
:canonical: evorl.algorithms.contrib.td3_v3.TD3TrainMetric.q2
:type: chex.Array
:value: >
   None

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3TrainMetric.q2
:parser: autodoc2_docstrings_parser
```

````

`````

`````{py:class} TD3V3Workflow(env: evorl.envs.Env, agent: evorl.agent.Agent, optimizer: optax.GradientTransformation, evaluator: evorl.evaluators.Evaluator, replay_buffer: evorl.replay_buffers.AbstractReplayBuffer, config: omegaconf.DictConfig)
:canonical: evorl.algorithms.contrib.td3_v3.TD3V3Workflow

Bases: {py:obj}`evorl.algorithms.offpolicy_utils.OffPolicyWorkflowTemplate`

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.TD3V3Workflow
:parser: autodoc2_docstrings_parser
```

````{py:method} learn(state: evorl.types.State) -> evorl.types.State
:canonical: evorl.algorithms.contrib.td3_v3.TD3V3Workflow.learn

````

````{py:method} name()
:canonical: evorl.algorithms.contrib.td3_v3.TD3V3Workflow.name
:classmethod:

````

````{py:method} step(state: evorl.types.State) -> tuple[evorl.metrics.MetricBase, evorl.types.State]
:canonical: evorl.algorithms.contrib.td3_v3.TD3V3Workflow.step

````

`````

````{py:function} make_mlp_td3_agent(action_space: evorl.envs.Space, norm_layer_type: str = 'none', critic_hidden_layer_sizes: tuple[int] = (256, 256), actor_hidden_layer_sizes: tuple[int] = (256, 256), discount: float = 0.99, exploration_epsilon: float = 0.5, policy_noise: float = 0.2, clip_policy_noise: float = 0.5, critics_in_actor_loss: str = 'first', normalize_obs: bool = False)
:canonical: evorl.algorithms.contrib.td3_v3.make_mlp_td3_agent

```{autodoc2-docstring} evorl.algorithms.contrib.td3_v3.make_mlp_td3_agent
:parser: autodoc2_docstrings_parser
```
````