SMPLOlympics: Sports Environments for Physically Simulated Humanoids

In the high jump environment, the humanoid jumps over a horizontal bar placed at a certain height without touching it and aims to reach a goal point that is 2 meters behind the bar. The bar is positioned following the setup of the official Olympic game. The high jump goal state ${\boldsymbol{s}^{\text{g-high\_jump}}_{t}}=(\boldsymbol{p}^{b}_{t},\boldsymbol{p}^{g}_{t})$ contains the positions of the bar $\boldsymbol{p}^{b}_{t}\in\mathbb{R}^{3}$ and the goal point $\boldsymbol{p}^{g}_{t}\in\mathbb{R}^{3}$. The reward is defined as $\boldsymbol{\mathcal{R}}^{\text{high jump}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-high\_jump}}_{t}})\triangleq w^{\text{p}}r^{\text{p}}_{t}+w^{\text{h}}r^{\text{h}}_{t}$. The position reward $r^{p}_{t}$ encourages the humanoid to go closer to the goal point. The height reward $r^{h}_{t}$ encourages the humanoid to jump higher. Training terminates when the humanoid is in contact with the bar, does not pass the bar, or falls to the ground before jumping. We also set up four bar heights for curriculum learning: 0.5m, 1m, 1.5m, and 2m.

Long jump is also set similar to the real-world setting, with a 20m runway followed by a jump area. Before the humanoid jumps, its feet should be behind the jump line. The goal state ${\boldsymbol{s}^{\text{g-long\_jump}}_{t}}\triangleq(\boldsymbol{p}^{s}_{t},\boldsymbol{p}^{l}_{t},\boldsymbol{p}^{g}_{t})$ includes the position of the starting point $\boldsymbol{p}^{s}_{t}\in\mathbb{R}^{3}$, jump line $\boldsymbol{p}^{l}_{t}\in\mathbb{R}^{3}$, and the goal $\boldsymbol{p}^{g}_{t}\in\mathbb{R}^{3}$. The training reward is defined as $\boldsymbol{\mathcal{R}}^{\text{long jump}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-long\_jump}}_{t}})\triangleq w^{\text{p}}r^{\text{p}}_{t}+w^{\text{v}}r^{\text{v}}_{t}+w^{\text{h}}r^{\text{h}}_{t}+w^{\text{l}}r^{\text{l}}_{t}$. The position reward $r^{\text{p}}_{t}$ encourages the humanoid to get closer to the goal, the velocity reward $r^{\text{v}}_{t}$ encourages larger running speed, and the height reward $r^{\text{h}}_{t}$ encourages higher jump. Finally, $r^{\text{l}}_{t}$ encourages jumping far.

In hurdling, the humanoid tries to reach a finishing line 110 meters ahead and needs to jump over 10 hurdles (each 1.067m high, placed 13.72m from the start, with subsequent hurdles spaced every 9.14m). The goal state is defined as ${\boldsymbol{s}^{\text{g-hurdling}}_{t}}\triangleq(\boldsymbol{p}^{h}_{t},\boldsymbol{p}^{f}_{t})$, where $\boldsymbol{p}^{h}_{t}\in\mathbb{R}^{10\times 3}$ and $\boldsymbol{p}^{f}_{t}\in\mathbb{R}^{3}$ includes the positions of these hurdles as well as the finish line. We define a simple reward function as $\boldsymbol{\mathcal{R}}^{\text{hurdling}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-hurdling}}_{t}})\triangleq r^{\text{distance}}_{t}$. $\boldsymbol{\mathcal{R}}^{\text{hurdling}}$ encourages the agent to run towards the finish line and clear each hurdle. Additionally, we employ a curriculum for hurdling, where the height of each hurdle is randomly sampled between 0 and 1.167 meters for each episode.

For golf, the humanoid’s right hand is replaced with a golf club measuring 1.14 meters. The driver of the golf club is simulated as a small box ($0.05\text{m}\times 0.025\text{m}\times 0.02\text{m}$). We incorporate a wave-like terrain with an amplitude of 0.5 meters in the golf environment, designed to mimic real-world grasslands. The golf goal is positioned to the left of the humanoid, at a distance ranging from 0 meters to 20 meters away. The goal state ${\boldsymbol{s}^{\text{g-golf}}_{t}}\triangleq(\boldsymbol{p}^{b}_{t},\boldsymbol{p}^{c}_{t},\boldsymbol{p}^{g}_{t},\boldsymbol{o}_{t})$ includes the ball position $\boldsymbol{p}^{b}_{t}\in\mathbb{R}^{3}$, club $\boldsymbol{c}^{b}_{t}\in\mathbb{R}^{3}$, goal position $\boldsymbol{p}^{g}_{t}\in\mathbb{R}^{3}$, and terrain height map $\boldsymbol{o}_{t}\in\mathbb{R}^{32\times 32}$. The reward is defined as $\boldsymbol{\mathcal{R}}^{\text{golf}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-golf}}_{t}})\triangleq w^{\text{p}}r^{\text{p}}_{t}+w^{\text{c}}r^{\text{c}}_{t}+w^{\text{g}}r^{\text{g}}_{t}+w^{\text{pred}}r^{\text{pred}}_{t}$, where the $r^{\text{p}}_{t}$ encourages the ball to move forward, $r^{\text{c}}_{t}$ encourages swinging the golf club to hit the ball, and $r^{\text{g}}_{t}$ encourages the ball to reach the goal. In addition, we predict the ball’s trajectory and provide a dense reward $r^{\text{pred}}_{t}$ based on the distance between the predicted landing point and the goal.

For javelin throw, we use SMPL-X humanoid with articulated fingers. The goal state is defined as ${\boldsymbol{s}^{\text{g-javelin}}_{t}}\triangleq(\boldsymbol{q}^{\text{obj}}_{t},\boldsymbol{p}^{r}_{t},\boldsymbol{p}^{h}_{t})$, where $\boldsymbol{q}^{\text{obj}}_{t}\in\mathbb{R}^{13}$, includes the position, orientation, linear, and angular velocity of the javelin. $\boldsymbol{p}^{r}_{t}$ and $\boldsymbol{p}^{h}_{t}$ are the positions of the root and right hand. The reward is defined as $\boldsymbol{\mathcal{R}}^{\text{javelin}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-javelin}}_{t}})\triangleq w^{\text{grab}}r^{\text{grab}}_{t}+w^{\text{js}}r^{\text{js}}_{t}+w^{\text{goal}}r^{\text{goal}}_{t}+w^{\text{s}}r^{\text{s}}_{t}$. The grab reward $r^{\text{grab}}_{t}$ encourages the right hand to grab the javelin. The javelin stability reward $r^{\text{js}}_{t}$ minimizes the javelin’s self-rotation. The goal reward $r^{\text{goal}}_{t}$ encourages the humanoid to throw the javelin further. The stability reward $r^{\text{s}}_{t}$ is to avoid large movements of the body.

For tennis, each humanoid’s right hand is replaced as an oval racket. We use the same measurement as a real tennis court and ball. We design two tasks: a single-player task where the humanoid trains to hit balls launched randomly, and a 1v1 mode where the humanoid plays against another humanoid. For both tasks, the goal state is defined as ${\boldsymbol{s}^{\text{g-tennis}}_{t}}\triangleq({\boldsymbol{p}^{\text{ball}}_{t}},{\boldsymbol{v}^{\text{ball}}_{t}},{\boldsymbol{p}^{\text{racket}}_{t}},{\boldsymbol{p}^{\text{tar}}_{t}}$, where ${\boldsymbol{p}^{\text{ball}}_{t}}\in\mathbb{R}^{3},{\boldsymbol{v}^{\text{ball}}_{t}}\in\mathbb{R}^{3},{\boldsymbol{p}^{\text{racket}}_{t}}\in\mathbb{R}^{3}$ and ${\boldsymbol{p}^{\text{tar}}_{t}}\in\mathbb{R}^{3}$, which includes the position and velocity of the ball, position of the racket and position of the target. The reward function for tennis is defined as $\boldsymbol{\mathcal{R}}^{\text{tennis}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-tennis}}_{t}})\triangleq w_{\text{p}}r^{\text{racket}}_{t}+w_{\text{b}}r^{\text{ball}}_{t}$. The racket reward $r^{\text{racket}}_{t}$ encourages the racket to reach the ball, and the ball reward $r^{\text{ball}}_{t}$ aims to successfully hit the ball into the opponent’s court, as close to the target as possible. For the single-player task, we shoot a ball from the opposite side from a random position and trajectory, simulating a ball hit by the opponent. The target ${\boldsymbol{p}^{\text{tar}}_{t}}$ is also randomly sampled. For the 1v1 scenario, we can either train models from scratch or initialize two identical single-player models as opponents, which can play back and forth.

For table tennis, each humanoid is equipped with a circular paddle (replacing the right hand) and plays on a standard table. Similar to tennis, we have the single-player task and the 1v1 task. Similarly, the goal state is defined as ${\boldsymbol{s}^{\text{g-tennis}}_{t}}\triangleq({\boldsymbol{p}^{\text{ball}}_{t}},{\boldsymbol{v}^{\text{ball}}_{t}},{\boldsymbol{p}^{\text{racket}}_{t}},{\boldsymbol{p}^{\text{tar}}_{t}})$. The reward function for table tennis is defined as $\boldsymbol{\mathcal{R}}^{\text{table tennis}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-table\_tennis}}_{t}})\triangleq w_{\text{p}}r^{\text{racket}}_{t}+w_{\text{b}}r^{\text{ball}}_{t}$. The paddle reward $r^{\text{racket}}_{t}$ is the same as tennis while we modify the $r^{\text{ball}}_{t}$ slightly to encourage more hits for table tennis.

For 1v1 fencing, each humanoid is equipped with a sword (replacing the right hand) and plays on a standard fencing field. The goal state is defined as ${\boldsymbol{s}^{\text{g-fencing}}_{t}}\triangleq({\boldsymbol{p}^{\text{opp}}_{t}},{\boldsymbol{v}^{\text{opp}}_{t}},{\boldsymbol{p}^{\text{sword}}_{t}}-{\boldsymbol{p}^{\text{opp-target}}_{t}},\|{\boldsymbol{{c}}_{t}}\|_{2}^{2},\|{\boldsymbol{{c}}^{\text{opp}}_{t}}\|_{2}^{2},\boldsymbol{b})$, which contains the opponent’s position body ${\boldsymbol{p}^{\text{opp}}_{t}}\in\mathbb{R}^{24\times 3}$, linear velocity ${\boldsymbol{v}^{\text{opp}}_{t}}\in\mathbb{R}^{24\times 3}$, the difference between target body position ${\boldsymbol{p}^{\text{opp-target}}_{t}}\in\mathbb{R}^{5\times 3}$ on the opponent and agent’s sword tip position ${\boldsymbol{p}^{\text{sword}}_{t}}$, normalized contract forces on the agent itself $\|{\boldsymbol{{c}}_{t}}\|_{2}^{2}\in\mathbb{R}^{24\times 3}$ and its opponent $\|{\boldsymbol{{c}}^{\text{opp}}_{t}}\|_{2}^{2}\in\mathbb{R}^{24\times 3}$, as well as the bounding box $\boldsymbol{b}\in\mathbb{R}^{4}$. To train the fencing agent, we define the fencing reward function as $\boldsymbol{\mathcal{R}}^{\text{fencing}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-fencing}}_{t}})\triangleq w_{\text{f}}r^{\text{facing}}_{t}+w_{\text{v}}r^{\text{vel}}_{t}+w_{\text{s}}r^{\text{strike}}_{t}+w_{\text{p}}r^{\text{point}}_{t}$. The facing $r^{\text{facing}}_{t}$ and velocity reward $r^{\text{vel}}_{t}$ encourage the agent to face and move toward the opponent. The strike reward $r^{\text{strike}}_{t}$ encourages the agent’s sword tip to get close to the target, while $r^{\text{point}}_{t}$ is the reward for getting in contact with the target. We use the pelvis, head, spine, chest, and torso as the target bodies. The episode terminates if either of the humanoids falls or steps out of bounds.

For boxing, we simulate two humanoids with sphere hands in a bounded arena. The goal state is similar to fencing: ${\boldsymbol{s}^{\text{g-boxing}}_{t}}\triangleq({\boldsymbol{p}^{\text{opp}}_{t}},{\boldsymbol{v}^{\text{opp}}_{t}},{\boldsymbol{{p}}_{t}^{\text{hand}}}-{\boldsymbol{p}^{\text{opp-target}}_{t}},\|{\boldsymbol{{c}}_{t}}\|_{2}^{2},\|{\boldsymbol{{c}}^{\text{opp}}_{t}}\|_{2}^{2})$ but without the bounding box information. The reward function and target body parts are also the same as fencing, though replacing the sword tip to the hands.

The soccer environment includes one or more humanoids, a ball, two goal posts, and the field boundaries. The field measures 32m $\times$ 20m. We support three tasks: penalty kicks, 1v1, and 2v2.

For penalty kicks, the humanoid is positioned 13 meters from the goal line, with the ball placed at a fixed spot 12 meters directly in front of the goal center. The objective is to kick the ball toward a randomly sampled target within the goal post. To achieve this, the controller is provided ${\boldsymbol{s}^{\text{g-kick}}_{t}}\triangleq({\boldsymbol{p}^{\text{ball}}_{t}},{\boldsymbol{\dot{q}}^{\text{ball}}_{t}},{\boldsymbol{p}^{\text{goal-post}}_{t}},{\boldsymbol{p}^{\text{goal-target}}_{t}})$, where ${\boldsymbol{p}^{\text{ball}}_{t}}\in\mathbb{R}^{3}$ is the ball position, ${\boldsymbol{\dot{q}}^{\text{ball}}_{t}}\in\mathbb{R}^{3}$ is the velocity and angular velocity, ${\boldsymbol{p}^{\text{goal-post}}_{t}}\in\mathbb{R}^{4}$ is the bounding box of the goal, and ${\boldsymbol{p}^{\text{goal-target}}_{t}}\in\mathbb{R}^{3}$ is the target location within the goal post. The reward is $\boldsymbol{\mathcal{R}}^{\text{soccer-kick}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-kick}}_{t}})\triangleq w^{\text{p2b}}r^{\text{p2b}}+w^{\text{b2g}}r^{\text{b2g}}+w^{\text{bv2g}}r^{\text{bv2g}}+w^{\text{b2t}}r^{\text{b2t}}-c^{\text{no-dribble}}_{t}$. Various rewards are designed to guide the character towards a run-and-kick motion. The player-to-ball ($r^{\text{p2b}}$) reward motivates the character to move towards the ball. The ball-to-goal reward ($r^{\text{b2g}}$) reduces the distance between the ball and the target. The ball-velocity-to-goal ($r^{\text{bv2g}}$) encourages a higher velocity of the ball toward the target. The ball-to-target ($r^{\text{b2t}}$) reward encourages a smaller distance between the target and the predicted landing spot of the ball based on its current position and velocity. Finally, a negative reward ($c^{\text{no-dribble}}_{t}$) is applied if the character passes the spawn position of the ball, which discourages dribbling and encourages kicking.

Beyond penalty kicks, we explore team-play dynamics, including 1v1 and 2v2. The controller is provided with a state defined as ${\boldsymbol{s}^{\text{g-soccer}}_{t}}\triangleq({\boldsymbol{p}^{\text{ball}}_{t}},{\boldsymbol{\dot{q}}^{\text{ball}}_{t}},{\boldsymbol{p}^{\text{goal-post}}_{t}},{\boldsymbol{p}^{\text{ally-root}}_{t}},{\boldsymbol{p}^{\text{opp-root}}_{t}})$, where ${\boldsymbol{p}^{\text{ally-root}}_{t}}\in\mathbb{R}^{3}$ and ${\boldsymbol{p}^{\text{opp-root}}_{t}}\in\mathbb{R}^{3}$ are the root positions of the ally and opponents (1 or 2). The controller is then trained using the following reward $\boldsymbol{\mathcal{R}}^{\text{soccer-match}}({\boldsymbol{s}^{\text{p}}_{t}},{\boldsymbol{s}^{\text{g-soccer}}_{t}})\triangleq w^{\text{p2b}}r^{\text{p2b}}+w^{\text{b2g}}r^{\text{b2g}}+w^{\text{bv2g}}r^{\text{bv2g}}+w^{\text{point}}r^{\text{point}}$, where $r^{\text{p2b}}$, $r^{\text{b2g}}$ and $r^{\text{bv2g}}$ are the same as in penalty kick. $r^{\text{b2g}}$ and $r^{\text{bv2g}}$ are zeroed out when the distance to the ball is greater than 0.5m. $r^{\text{point}}$, the scoring a goal, provides a one-time bonus and or penalty for goals. Notice that this is a rudimentary reward design compared to prior art [8] and serves as a starting point for further development.

Our basketball environment is set up similarly to the soccer environment except for using the SMPL-X humanoid. The court measures 29m $\times$ 15m, with a 3m high hoop. We also introduce the task of free-throw, where the humanoid begins at a distance of 4.5 meters from the hoop with the ball initially positioned close to its hands. The objective is to successfully throw the basketball into the hoop. The goal state for this task is defined similarly to that of the soccer penalty kicks, with the distinction being the prohibition of foot-to-ball contact to maintain basketball rules.

In competitive sports environments, we implement a basic adversarial self-play mechanism where two policies, initialized randomly, compete against each other to optimize their rewards. We adopt an alternating optimization strategy from [29], where one policy is frozen while the other is trained. This encourages each policy to develop offensive and defensive strategies, contributing to more competitive behavior, as observed in boxing and fencing (supplement site).

We first evaluate track and field sports, including long jump, high jump, hurdling, and javelin throwing. For these sports, SOTA pose estimation methods fail to estimate coherent motion and global root trajectory as players and cameras are both fast-moving. Thus, we utilize a subset of the AMASS dataset containing locomotion data [21] as reference motions. Since PULSE is pretrained on AMASS, we exclude PULSE + AMP from these tests. Table 1 summarizes the quantitative results of different methods. In long jump, AMP fails entirely, often walking slowly to the jump line without a forward leap. This failure occurs because the policy prioritizes discriminator rewards over task completion. If the task is too hard, the policy will use simple motion (such as standing still) to maximize the discriminator reward instead of trying to complete the task. In contrast, PPO, while capable of jumping great distances, exhibits unnatural motions. PULSE successfully executes jumps with human-like motion, but lacks the specialized skills for top-tier records due to the absence of corresponding motion data in AMASS. The high jump displays similar patterns: PPO achieves impressive heights but with unnatural movements while AMP struggles to reconcile adversarial and task rewards. Surprisingly, as shown in Figure 2, PULSE successfully adopts a Fosbury flop approach without specific rewards to encourage this technique, likely leveraging breakdance skills. For hurdling, AMP completely fails, stopping before the first hurdle. PPO bounces energetically over each obstacle as shown in Figure 2, but sometimes falls and fails to complete the race, with an average success rate of just over 50% and an average distance of less than 110m. PULSE facilitates natural clearance of hurdles, and completes races in 17.76 seconds, a competitive time compared to human standards. Javelin throwing poses similar challenges: PPO uses inhuman strategies, AMP struggles with balancing rewards, and PULSE adopts human-like strategies but lacks specific skills for record-setting performance.

For sports including golf, tennis, table tennis, and soccer penalty kick, we utilize processed motion from videos as demonstrations for AMP and PULSE+AMP. The results are reported in Table 2 and Fig. 3. In tennis, AMP demonstrates superior performance in terms of average hits; however, returned balls often land far from the intended targets. This is because prolonged rallies increase discriminator rewards, leading AMP to ignore task rewards. Notably, AMP exhibits inhuman motions at the moment of ball contact and reverts to natural movements when preparing for the next hit as shown in Fig. 3. This behavior underscores a reward conflict between balancing task and discriminator rewards. PPO plays tennis in an unnatural way, while PULSE and PULSE + AMP show similar performance. In table tennis, PPO achieves impressive error distances, but struggles with consistency and often fails to return second shots. We observe video data proves particularly beneficial for table tennis. PULSE+AMP records significantly higher hit averages with reasonable error distances. Table tennis requires quick reactions within a short time, which the pre-trained PULSE model supports by providing necessary motor skills, enhanced by video data that guide the learning of proper stroke techniques. For golf, penalty kicks, and free throws, the “initiating contact with an object" part makes them challenging. Here, only PULSE and PULSE+AMP manage to solve the three tasks effectively and consistently, leveraging PULSE’s latent space for effective exploration. The design of these tasks often results in a sparse exploration phase where triggering penalty rewards, such as $c^{\text{no-dribble}}_{t}$ for moving past the ball’s initial position. The AMP reward also negatively affects training penalty kick, as the human demonstration contains other soccer motions such as running and dribbling, and the policy finds them easier to learn and exploit.

the curriculum using PULSE. We can see that without curriculum, high jump and hurdling both fail to solve the task. This is due to the policy not being able to obtain any reward facing challenging heights of bars and hurdles and the policy gets stuck in the local minima.