Soft Actor-Critic (SAC) and Deep Q Learning (DQL)

DQL OVERVIEW

Store experiences in a replay buffer

Collect transitions through interaction

Update Eval-Net using sampled experiences and back propagation

Periodically copy Eval-Net weights to Target-Net

maximize entropy (exploration) 

while also trying to maximize the reward

Actor: Policy that determines actions given the current state.

Critic:

• Used to estimate state-action values.

• Two critics are trained independently to avoid overestimation of Q-Values

Replay Buffer: Stores information on state, action, reward, and next state.

The quadrupeds XML were first modeled and then transferred to MuJoCo (physics simulator) to perform RL training

PATRIQ robot XML model