Update README.md

README.md

@@ -1,3 +1,28 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+# DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences
+
+### [Project Page](https://sled-group.github.io/driVLMe/) | [Paper](https://arxiv.org/abs/2406.03008) | [Video](https://youtu.be/Ep5fYLGkmsg)  | [Code](https://github.com/sled-group/driVLMe/tree/main)
+
+Yidong Huang, Jacob Sansom, Ziqiao Ma, Felix Gervits, Joyce Chai  
+University of Michigan, ARL  
+IROS 2024
+
+
+You can also download the pretrained checkpoints from [this link](https://www.dropbox.com/scl/fo/neqjdlhohygoa0wrv4uuy/AAjarkE6WY6sKt4LoAfyZ3c?rlkey=e0yvw6g1j8qqdp63vhgi0722d&st=tp2w6h3f&dl=0)
+
+To run the open-loop evaluation, we can use the command 
+```shell
+python drivlme/single_video_inference_SDN.py --model-name  /nfs/turbo/coe-chaijy-unreplicated/pre-trained-weights/LLaVA/LLaVA-7B-Lightening-v1-1/ --projection_path ./DriVLMe_model_weights/bddx_pretrain_ckpt/mm_projector.bin --lora_path  ./DriVLMe_model_weights/DriVLMe/ --json_path datasets/SDN_test_actions.json --video_root videos/SDN_test_videos/ --out_path SDN_test_actions.json
+
+python evaluation/physical_action_acc.py
+```
+for NfD task and 
+```shell
+python drivlme/single_video_inference_SDN.py --model-name  /nfs/turbo/coe-chaijy-unreplicated/pre-trained-weights/LLaVA/LLaVA-7B-Lightening-v1-1/ --projection_path ./DriVLMe_model_weights/bddx_pretrain_ckpt/mm_projector.bin --lora_path  ./DriVLMe_model_weights/DriVLMe/ --json_path datasets/SDN_test_conversations.json --video_root videos/SDN_test_videos/ --out_path SDN_test_conversations.json
+
+python evaluation/diag_action_acc.py
+```
+for RfN task.