From 316ade4101d940628d627c46f83c93b07bc538eb Mon Sep 17 00:00:00 2001
From: Yangge Li <li213@illinois.edu>
Date: Mon, 24 Oct 2022 15:00:32 -0500
Subject: [PATCH] start working on tutorial

---
 README.txt | 129 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 129 insertions(+)
 create mode 100644 README.txt

diff --git a/README.txt b/README.txt
new file mode 100644
index 00000000..42127f66
--- /dev/null
+++ b/README.txt
@@ -0,0 +1,129 @@
+This is the repeatability evaluation package for the tool paper "Verse: A Python library for reasoning about multi-agent hybrid system scenarios"
+
+###########################
+
+The link to the artifact on Google drive is: 
+
+The link to the artifact on Figshare is:
+
+The link to the source code git repository for Verse is: 
+
+###########################
+
+The run-time for regenerating data in Table 1 in the paper is approximately 2650s (~44min).
+
+The run-time for regenerating data in Simulation section of Table 2 in the paper is approximately 174s (~3min).
+
+The run-time for regenerating data in Reach section of Table 2 in the paper is approximately ?s (~?min).
+
+###########################
+
+To replicate our experiments from the paper, please run following commands from the root directory of the artifact. 
+1. For results in Table 1 Run command 
+    python3 exprs.py 
+The results for all experiments in Table 1 will be generated as below at the end of execution. 
+    2 & Q & M6 & DryVR & No & 8 & 53.82\\
+    2 & Q & M5 & DryVR & No & 5 & 17.76\\
+    2 & Q & M5 & NeuReach & No & 5 & 475.47\\
+    3 & Q & M5 & DryVR & No & 7 & 38.26\\
+    7 & C & M2 & DryVR & No & 37 & 343.26\\
+    3 & C & M1 & DryVR & No & 5 & 22.51\\
+    3 & C & M3 & DryVR & No & 4 & 34.12\\
+    3 & C & M4 & DryVR & No & 7 & 116.65\\
+    3 & C & M1 & DryVR & Yes & 5 & 29.84\\
+    2 & C & M1 & DryVR & No & 5 & 14.66\\
+    2 & C & M1 & NeuReach & No & 5 & 585.19\\
+
+2. For results in Table 2,
+    Simulation: Run command 
+        python3 inc-expr.py s
+    The results for the experiments in Table 2 section Simulation will be generated as below at the end of execution
+        & repeat & 45 & 16.92 & 430 & 1.05 & 438 & 3.83 & 83.33 \\
+        & change init & 24 & 14.93 & 431 & 12.98 & 436 & 4.07 & 75.91 \\
+        & change ctlr & 45 & 16.1 & 431 & 8.49 & 438 & 4.38 & 78.19 \\
+
+    Verification: Run command 
+        python3 inc-expr.py v 
+    The results for the experiments in Table 2 section Verification will be generated as below at the end of execution
+        # TODO:
+        
+3. To reproduce Fig. 1,
+    Center: Run command 
+        python3 demo/tacas2023/exp1/exp1_sim.py p 
+    Verse will automatically open a browser window to show the resulting plot. 
+
+    Right: Run command 
+        python3 demo/tacas2023/exp1/exp1.py p
+
+4. To reproduce Fig. 4, from left to right 
+    (1) Run command 
+        python3 python3 demo/tacas2023/exp9/exp9_dryvr.py pl
+
+    (2)-(3) Run command 
+        python3 demo/tacas2023/exp9/exp9_dryvr.py pc
+    Verse will automatically open two browser windows to show the two resulting plots.
+
+    (4) Run command 
+        python3 demo/tacas2023/exp10/exp10_dryvr.py p 
+    
+5. To reproduce Fig. 6, 
+    Left: Run command 
+        python3 demo/tacas2023/exp2/exp2_straight.py p 
+    Verse will automatically open a browser window to show the resulting plot. 
+    
+    Center: Run command 
+        python3 demo/tacas2023/exp2/exp2_curve.py p 
+    Verse will automatically open a browser window to show the resulting plot. 
+    
+    Right: Run command 
+        python3 demo/tacas2023/exp4/exp4.py p
+    Verse will automatically open a browser window to show the resulting plot. 
+
+6. To reproduce Fig. 8 in Appendix, run command 
+    python3 demo/tacas2023/exp5/exp5.py p
+Verse will automatically open a browser window to show the resulting plot. The right figure is a zoom in of the left figure
+
+
+7. To reproduce Fig. 9 in Appendix, 
+    Left: Run command 
+        python3 demo/tacas2023/exp6/exp6_neureach.py p
+    Verse will automatically open a browser window to show the resulting plot. 
+
+    Right: Run command 
+        python3 demo/tacas2023/exp6/exp6_dryvr.py p 
+    Verse will automatically open a browser window to show the resulting plot. 
+
+8. To reproduce Fig. 10 in Appendix, run command  
+    python3 demo/tacas2023/exp7/exp7.py p
+Verse will automatically open two browser windows to show the plots.
+
+9. To reproduce Fig. 11 in Appendix, run command 
+    python3 demo/tacas2023/exp3/exp3.py p
+Verse will automatically open a browser window to show the resulting plot. 
+
+10. To reproduce Fig. 12 in Appendix 
+    Left: Run command 
+        python3 demo/tacas2023/exp11/inc-expr.py sbp
+    Verse will automatically open a browser window to show the resulting plot. 
+
+    Right: Run command 
+        python3 demo/tacas2023/exp11/inc-expr.py vbp
+    Verse will automatically open a browser window to show the resulting plot. 
+
+11. To reproduce Fig. 15 in Appendix 
+    Left: Run command 
+        python3 
+    Verse will automatically open a browser window to show the resulting plot. 
+
+    Right: Run command 
+        python3 
+    Verse will automatically open a browser window to show the resulting plot. 
+
+12. To reproduce Fig. 16 in Appendix 
+    Left: Run command 
+        python3 
+    Verse will automatically open a browser window to show the resulting plot. 
+
+    Right: Run command 
+        python3 
+    Verse will automatically open a browser window to show the resulting plot. 
-- 
GitLab