Omniverse RTX Lidar Guide

Step 1: What is RTX Lidar?

• Lidar measures distances to surrounding objects by emitting laser beams.
• RTX Lidar uses NVIDIA’s RTX (ray tracing) technology to simulate a lidar sensor in a virtual 3D environment as it behaves in the real world.
• It is used in robots, autonomous vehicles, drones, etc. for environment perception, obstacle detection, and mapping.

[Robot]---(Laser)---[Wall]
| ↑
|<---Distance----|

RTX Lidar cannot perceive the “thing-in-itself (Ding an sich).” Instead, it provides processed data in a perceivable format (phenomenon). In other words, RTX Lidar does not see reality “as is,” but reconstructs the world into a form we can interpret within the frameworks we define (parameters, annotators, writers).

Step 2: The Four Core Building Blocks of RTX Lidar

The RTX Lidar system consists of four core file types.

[1. Asset USD File] ←--→ [2. USDA Parameter File]
↓ ↓
[3. Registered Config] ←--→ [4. JSON Template]
↓ ↓
[LidarRtx Validation] ←--→ [Runtime Conversion]
↓ ↓
[USD Property Integration]
↓
[RTX Engine Simulation]

End-to-End Data Flow Overview

1. Asset USD: 3D model + variant definitions
2. USDA Parameters: Precise sensor specs for each variant
3. Registered Config: List of supported models
4. JSON Template: User-friendly settings + external tool compatibility

→ LidarRtx integrates these and runs real-time simulation on the RTX engine.

Step 3: In-Depth Look at the Four Core Files

3.1 Asset USD File (3D Model + Variant Definitions)

Role: Integrates the lidar sensor’s visual representation and multiple configuration options in a single file.

Example Location: https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0/Isaac/Sensors/Ouster/OS0/OS0.usd

Structure:

OS0.usd
├── [3D Mesh] Visual model of the lidar sensor
├── [Material] Texture and material info
└── [Variant Set] Multiple configuration options
├── "OS0_REV6_128ch10hz512res" # 512 resolution
├── "OS0_REV6_128ch10hz1024res" # 1024 resolution
├── "OS0_REV6_128ch10hz2048res" # 2048 resolution
└── ... (more variants)

Key Characteristics:

• Multiple sensor settings in one file (distinguished by variants)
• Accurate, manufacturer-grade 3D model
• Cloud distribution via Nucleus server
• Versioning (e.g., REV6, REV7)

3.2 USDA Parameter File (Precise Sensor Specs)

Role: Defines precise physical sensor parameters per variant in USD format.

Example Location:

https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0/Isaac/Sensors/Ouster/OS0/OS0_REV6_128_10hz_512_resolution.usda

#usda 1.0
(
defaultPrim = "OS0_REV6_128_10hz_512_resolution"
doc = "Generated from OS0_REV6_128ch10hz512res.json"
)

def OmniLidar "OS0_REV6_128_10hz_512_resolution" (
prepend apiSchemas = ["OmniSensorGenericLidarCoreAPI"]
)
{
# Basic sensor attributes
float omni:sensor:Core:avgPowerW = 0.002
float omni:sensor:Core:nearRangeM = 0.3
float omni:sensor:Core:farRangeM = 50
uint omni:sensor:Core:numberOfChannels = 128
uint omni:sensor:Core:scanRateBaseHz = 10

# Beam angles (128 channels)
float[] omni:sensor:Core:emitterState:s001:azimuthDeg = [
4.23, 1.41, -1.4, -4.21, 4.22, 1.42, -1.41, -4.22,
4.23, 1.41, -1.42, -4.21, 4.23, 1.42, -1.4, -4.2,
# ... total 128
]
float[] omni:sensor:Core:emitterState:s001:elevationDeg = [
44.5, 43.8, 43.1, 42.4, 41.7, 41, 40.3, 39.59,
38.89, 38.19, 37.49, 36.79, 36.09, 35.39, 34.69,
# ... total 128
]

# Timing
uint[] omni:sensor:Core:emitterState:s001:fireTimeNs = [
762, 2287, 3812, 5337, 6862, 8387, 9912, 11437,
# ... total 128
]

# Channel mapping
uint[] omni:sensor:Core:emitterState:s001:channelId = [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
# ... total 128
]

# Error model
float omni:sensor:Core:azimuthErrorMean = 0
float omni:sensor:Core:azimuthErrorStd = 0.01
float omni:sensor:Core:elevationErrorMean = 0
float omni:sensor:Core:elevationErrorStd = 0.01

# Other properties...
float omni:sensor:Core:rangeAccuracyM = 0.03
float omni:sensor:Core:rangeResolutionM = 0.001
token omni:sensor:Core:rayType = "IDEALIZED"
uint omni:sensor:Core:reportRateBaseHz = 5120
token omni:sensor:Core:rotationDirection = "CW"
token omni:sensor:Core:scanType = "ROTARY"
string omni:sensor:modelName = "OS0_REV6_128_10hz_512_resolution"
float omni:sensor:tickRate = 10
}


Parameter Categories:

Key Characteristics:

• Accurate values based on manufacturer datasheets
• Directly compatible with the RTX engine
• Variant-specific optimized parameters
• Fine-grained definitions like per-beam angles for 128 beams

3.3 Registered Lidar Config File (Supported Model List)

Role: Defines all officially supported models and variants for RTX Lidar.

File Location: source/extensions/isaacsim.sensors.rtx/python/impl/supported_lidar_configs.py

Key Characteristics:

• Official model list validated by NVIDIA
• Variant validation: only supported configurations are allowed
• Organized by manufacturer: Ouster, SICK, NVIDIA, SLAMTEC, etc.
• Custom additions possible: Even if not listed, you can create custom lidars via USD assets or JSON.

Examples of Officially Supported Lidar Models and Variants

By Manufacturer:

Manufacturer Highlights:

Variant Naming Rules:

• Ouster: {model}_{revision}_{channels}ch{freq}hz{resolution}res
• SICK: Normal_{index} or Extended_{index}
• NVIDIA: Functional names (Example, Debug, Simple, etc.)

Full configs Code:

# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Configuration definitions for supported Lidar sensors in Isaac Sim.

This module defines the supported Lidar sensor configurations and their variants
that can be used with the RTX sensor system. It includes configurations for various
manufacturers.
"""

# Expected name of Lidar prim variant sets
SUPPORTED_LIDAR_VARIANT_SET_NAME = "sensor"

# Map supported Lidar asset paths to their variants
SUPPORTED_LIDAR_CONFIGS = {
"/Isaac/Sensors/HESAI/XT32_SD10/HESAI_XT32_SD10.usd": set(),
"/Isaac/Sensors/NVIDIA/Debug_Rotary.usda": set(),
"/Isaac/Sensors/NVIDIA/Example_Rotary_2D.usda": set(),
"/Isaac/Sensors/NVIDIA/Example_Rotary.usda": set(),
"/Isaac/Sensors/NVIDIA/Example_Solid_State.usda": set(),
"/Isaac/Sensors/NVIDIA/Simple_Example_Solid_State.usda": set(),
"/Isaac/Sensors/Ouster/OS0/OS0.usd": {
"OS0_REV6_128ch10hz1024res",
"OS0_REV6_128ch10hz2048res",
"OS0_REV6_128ch10hz512res",
"OS0_REV6_128ch20hz1024res",
"OS0_REV6_128ch20hz512res",
"OS0_REV7_128ch10hz1024res",
"OS0_REV7_128ch10hz2048res",
"OS0_REV7_128ch10hz512res",
"OS0_REV7_128ch20hz1024res",
"OS0_REV7_128ch20hz512res",
},
"/Isaac/Sensors/Ouster/OS1/OS1.usd": {
"OS1_REV6_128ch10hz1024res",
"OS1_REV6_128ch10hz2048res",
"OS1_REV6_128ch10hz512res",
"OS1_REV6_128ch20hz1024res",
"OS1_REV6_128ch20hz512res",
"OS1_REV6_32ch10hz1024res",
"OS1_REV6_32ch10hz2048res",
"OS1_REV6_32ch10hz512res",
"OS1_REV6_32ch20hz1024res",
"OS1_REV6_32ch20hz512res",
"OS1_REV7_128ch10hz1024res",
"OS1_REV7_128ch10hz2048res",
"OS1_REV7_128ch10hz512res",
"OS1_REV7_128ch20hz1024res",
"OS1_REV7_128ch20hz512res",
},
"/Isaac/Sensors/Ouster/OS2/OS2.usd": {
"OS2_REV6_128ch10hz1024res",
"OS2_REV6_128ch10hz2048res",
"OS2_REV6_128ch10hz512res",
"OS2_REV6_128ch20hz1024res",
"OS2_REV6_128ch20hz512res", "OS2_REV7_128ch10hz1024res", "OS2_REV7_128ch10hz2048res", "OS2_REV7_128ch10hz512res", "OS2_REV7_128ch20hz1024res", "OS2_REV7_128ch20hz512res", }, "/Isaac/Sensors/Ouster/VLS_128/Ouster_VLS_128.usd": set(), "/Isaac/Sensors/SICK/microScan3/SICK_microScan3.usd": { "Normal_1", "Normal_2", "Normal_4", "Normal_5", "Normal_6", }, "/Isaac/Sensors/SICK/multiScan136/SICK_multiScan136.usd": set(), "/Isaac/Sensors/SICK/multiScan165/SICK_multiScan165.usd": set(), "/Isaac/Sensors/SICK/nanoScan3/SICK_nanoScan3.usd": { "CAAZ30AN1", }, "/Isaac/Sensors/SICK/picoScan150/SICK_picoScan150.usd": { "Normal_1", "Normal_2", "Normal_3", "Normal_4", "Normal_6", "Normal_7", "Normal_8", "Normal_9", "Normal_10", "Normal_11", }, "/Isaac/Sensors/SICK/TIM781/SICK_TIM781.usd": set(), "/Isaac/Sensors/Slamtec/RPLIDAR_S2E/Slamtec_RPLIDAR_S2E.usd": set(), "/Isaac/Sensors/ZVISION/ZVISION_ML30S.usda": set(), "/Isaac/Sensors/ZVISION/ZVISION_MLXS.usda": set(),}

3.4 JSON Template (User-Friendly Configuration)

File Location: source/extensions/isaacsim.sensors.rtx/data/lidar_configs/NVIDIA/Simple_Example_Solid_State.json

Role: Provides a user-friendly format and compatibility with external tools.

Full JSON Template:

{
"class": "sensor",
"type": "lidar",
"name": "Simple Solid State",
"driveWorksId": "GENERIC",
"profile":
{
"scanType": "solidState",
"intensityProcessing": "normalization",
"rotationDirection": "CW",
"rayType": "IDEALIZED",
"nearRangeM": 1.0,
"farRangeM": 200.0,
"effectiveApertureSize": 0.0145,
"focusDistM": 0.16,
"rangeResolutionM": 0.004,
"rangeAccuracyM": 0.025,
"avgPowerW": 0.002,
"minReflectance": 0.1,
"minReflectanceRange": 270.0,
"wavelengthNm": 1550.0,
"pulseTimeNs": 6,
"maxReturns": 2,
"scanRateBaseHz": 30.0,
"reportRateBaseHz": 30,
"numberOfEmitters": 12,
"numberOfChannels": 12,
"rangeCount": 1,
"ranges": [
{"min": 0.5, "max": 300}
],
"azimuthErrorMean": 0.0,
"azimuthErrorStd": 0.025,
"elevationErrorMean": 0.0,
"elevationErrorStd": 0.025,
"stateResolutionStep": 1,
"numLines": 3,
"numRaysPerLine": [4, 4, 4],
"emitterStateCount": 1,
"emitterStates": [
{
"azimuthDeg": [
-2, -1, 1, 2,
-1.5, -0.5, 0.5, 1.5,
-2.4, -1.4, 1.4, 2.4
], "_commentOnAzimuth": "Flat Scan is expecting equal angles between each emitter in the line it uses",
"elevationDeg": [
-8, -8, -8, -8,
0, 0, 0, 0,
8, 8, 8, 8
], "_commentOnElevation": "Flat Scan use the line with elevation nearest 0 as it's first line entry",
"fireTimeNs": [
0,3300000,6600000,9900000,
13300000,16600000,19900000,23300000,
26600000,29900000,31500000,33300000],
"channelId": [
0,1,2,3,
4,5,6,7,
8,9,10,11],
"rangeId": [
0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0
],
"bank": [
11, 10, 9, 8,
7, 6, 5, 4,
3, 2, 1, 0
] } ], "intensityMappingType": "LINEAR" }}

Parameter Details

By Parameter Group:

Step 4: Relationships and Data Flow Among the Four Files

4.1 Relationship Diagram

[1. Asset USD] ←--→ [2. USDA Parameters]
↓ ↓
[3. Registered Config] ←--→ [4. JSON Template]
↓ ↓
[LidarRtx Validation] ←--→ [Runtime Conversion]
↓ ↓
[USD Property Integration]
↓
[RTX Engine Simulation]


4.2 Actual Data Flow

Scenario 1: Using a USD Asset

1. User: LidarRtx(config="/Isaac/Sensors/Ouster/OS0/OS0.usd", variant="OS0_REV6_128ch10hz512res")
2. Validation: Check support in supported_lidar_configs.py
3. Load: Pull the corresponding USDA parameters for the variant from the Asset USD
4. Apply: USD properties are applied directly to the RTX engine
5. Simulation: RTX ray tracing runs

Scenario 2: Using a JSON Template

1. User: LidarRtx(config="Simple_Example_Solid_State")
2. Lookup: Search JSON in data/lidar_configs/
3. Convert: JSON → USD properties (SensorNodeUtils.cpp)
4. Apply: Converted USD properties applied to the RTX engine
5. Simulation: RTX ray tracing runs

4.3 Priority and Selection Criteria

Processing Priority:

1. USD Asset + Variant (highest priority)
2. JSON Template (applied after conversion)
3. Defaults (fallback)

Recommended by Use Case:

Summary

RTX Lidar is a system where four core components work together:

1. Asset USD: 3D model + variant structure
2. USDA Parameters: Precise sensor specs (manufacturer-level)
3. Lidar Config: Validation list of supported models
4. JSON Template: User-friendly configuration

The LidarRtx class integrates these and provides real-time ray-tracing lidar simulation on the RTX engine. Depending on your goal, you can flexibly choose between high-fidelity hardware simulation (USD) and rapid prototyping (JSON).

Step 5: Analyzing at the Code Level

5.1 Prim (OmniLidar Prim)

• A Prim is the entity representing the sensor in USD (Universal Scene Description) within Omniverse.
• Creating a LidarRtx internally creates an OmniLidar Prim.

Code Example (Creating a Prim)

# Specify prim_path when creating LidarRtx
# Location: exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py
lidar = LidarRtx(prim_path="/World/lidar", ...)

• Internally, the prim is created/validated like this.

# Location: exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py
if is_prim_path_valid(prim_path):
...
else:
_, sensor = omni.kit.commands.execute(
"IsaacSensorCreateRtxLidar",
translation=Gf.Vec3d(...),
orientation=Gf.Quatd(...),
path=prim_path,
...)


5.2 LidarRtx Class

• The core Python class controlling all RTX Lidar behavior.
• Handles sensor creation, initialization, parameter settings, attaching annotators/writers, and data retrieval.

Code Example (Class Declaration and Key Methods)

# Location: exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py
class LidarRtx(BaseSensor):
...
def __init__(self, prim_path: str, name: str = "lidar_rtx", ...):
...
def initialize(self, ...):
...
def attach_annotator(self, annotator_name: str):
...
def attach_writer(self, writer_name: str):
...
def get_current_frame(self) -> dict:
...


• Usage example:

# Location: exts/isaacsim.sensors.rtx/docs/rtx_lidar.md (usage)
lidar = LidarRtx(prim_path="/World/lidar", position=[0,0,1], name="lidar")
lidar.initialize()


5.3 Annotator (Data Generator)

• An annotator attaches to LidarRtx to generate sensor data (point clouds, flat scans, etc.).
• Multiple annotators can be attached simultaneously to generate various outputs at once.

Code Example (Attaching Annotators)

# Location: exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py
lidar.attach_annotator("GenericModelOutputLidarPointAccumulator") # 3D point cloud
lidar.attach_annotator("IsaacComputeRTXLidarFlatScan") # 2D flat scan (for 2D lidars)

• Internally, annotators receive raw data from the RTX engine and process it.
• Annotator implementations:
  • exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py (attach_annotator)
  • Actual annotator logic is handled by Omniverse C++/USD nodes or Replicator annotators.

5.4 Writer (Visualizer/Exporter)

• A writer visualizes or saves the data produced by annotators.

Code Example (Attaching Writers)

# Location: exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py
lidar.attach_writer("RtxLidarDebugDrawPointCloud") # visualize point cloud

• Writers either draw data on screen or export it to files.
• Writer implementations:
  • exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py (attach_writer)
  • Actual writer logic is handled by Omniverse C++/USD nodes or Replicator writers.

5.5 End-to-End Data Flow Example

# Location: exts/isaacsim.sensors.rtx/docs/rtx_lidar.md (usage)
from isaacsim.sensors.rtx import LidarRtx
lidar = LidarRtx(prim_path="/World/lidar", position=[0,0,1], name="lidar")
lidar.initialize()
lidar.attach_annotator("GenericModelOutputLidarPointAccumulator")
lidar.attach_writer("RtxLidarDebugDrawPointCloud")

frame = lidar.get_current_frame()
print(frame["pointCloud"]) # 3D points generated by the annotator

• The core implementation of the data flow resides in exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/lidar_rtx.py .