https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/ Recent content in RStudio Workbench on Handbook Hugo -- gohugo.io https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/1_good_to_know/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/1_good_to_know/ Connecting through ssh File Editors Disk Usage Effective Bulk File Transfers General things good to know before doing any Workbench management related tasks. Connecting through ssh Terminal command to connect to an instance: ssh -i <key pem file> ubuntu@<instance public IP v4 DNS> Example: ssh -i RStudio_Workbench.pem ubuntu@ec2-35-173-234-97.compute-1.amazonaws.com Instance public IP v4 DNS can be found at the instance page under the network tab. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2_system_configuration/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2_system_configuration/ Hardware EC2 EC2 role EC2 reservation FSx Lustre setup Creating a file system Installing the Lustre Client Mounting the file system RStudio Workbench installation Network settings Mapping web domain to an elastic IP https Application Load Balancer Network Load Balancer Overview Hardware Currently our Workbench setup uses one EC2 server with a mounted FSx Lustre storage. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2-1_system_configuration_disregarded/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2-1_system_configuration_disregarded/ This page provides details about deprecated system configuration options. While we no longer use them, this information might come in handy in the future. Multi-attach EBS Installation Auto-Enabled IO Backups EFS Multi-attach EBS Installation Here’s the guide on creating a multi-attach compatible EBS volume: link. Select the io2 type. For size I chose 500 Gb, for IOPS - 1000. Size can be scaled up in the future (not down) and IOPS can be increased or decreased. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/3_users/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/3_users/ Clumsy way (optional) Google sign in setup Adding users Admin users Removing users This chapter covers user management and Google OpenID sign-in setup. Clumsy way (optional) Here’s a command to add a user to Ubuntu server: sudo useradd -m -d /pfs/home/<user> <user> sudo passwd <user> # will ask to enter a password Once you’ve added a user it should be possible to login into Workbench using this user’s Ubuntu credentials. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/4_global_settings/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/4_global_settings/ tmp folder Rprofile.site This chapter covers global system settings that will be applied to all users. tmp folder By default Ubuntu uses “/tmp” folder for temporary storage. In our current setup we use rather small internal ec-2 storage. So there’s a certain risk that this storage will be filled to capacity, especially if somebody happens to be doing Stan modeling. Therefore it makes sense to set the default temporary storage to a folder on the mounted persistent drive. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/5_r_libraries/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/5_r_libraries/ R library folder renv folders Initial package installation This chapter is dedicated to our R library setup. We want to store R libraries along with renv cache on the mounted persistent drive. In the future it may be useful to keep multiple versions of R installed. R library folder We will use R installations on ec-2 internal storage with global shared libraries on the mounted drive. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/6_package_installation_issues/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/6_package_installation_issues/ Problems with *.so libraries Cannot open the connection cmdstanr Other individual cases Problems with *.so libraries If you see an error like this: Error: package or namespace load failed for ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/pfs/R/libs/4.2.0/sf/libs/sf.so': libproj.so.15: cannot open shared object file: No such file or directory Most likely it means that a package was installed from a binary. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/7_testing_server/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/7_testing_server/ System state Turn off Open ID Known issues To properly try potential Workbench server updates you have to create a testing server. It’s not a good idea to test on our main server! Although for the tests to be as relevant as possible it’s better to have your testing environment closely resemble the production one. In our case in the first place this implies the state system drive of the server instance. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/8_job_launcher/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/8_job_launcher/ Introduction Setup Overview Considered but Unchosen Setup Options Slurm instead of Kubernetes Fargate for EKS Other storage options Prerequisites Tutorials Cloud9 Setup Creating EKS Cluster with Primary Node Group Karpenter RStudio service account FSx PVC RStudio configuration rserver.conf launcher-mounts launcher.conf launcher.kubernetes.profiles.conf launcher.kubernetes.conf kubernetes.launcher.token RStudio image Troubleshooting Launcher logs LD libraries Introduction The RStudio Server typically meets our needs, but occasionally running complex models can overload it, affecting user experience. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/9_monorepo_mirror/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/9_monorepo_mirror/ We are accustomed to having a separate copy of our monorepo for each project we work on. However, since our repository is quite large, doing git clone every time can be incredibly slow. To address this issue, we’ve set up an on-server monorepo mirror. This is a centralized copy that is regularly synchronized with GitHub. This allows us to quickly create new copies from the mirror instead of cloning the entire repository each time. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/10_submitting_diagnostics_reports/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/10_submitting_diagnostics_reports/ When submitting a support ticket to Posit Support regarding an issue with Workbench, it is advisable to include a diagnostics report. Typically, the support team will request this report if it is not initially provided. Below, we outline an efficient method for submitting this report. Access your server instance via SSH. Execute sudo rstudio-server run-diagnostics. You’ll see a message indicating the diagnostics report has been generated, along with its file path, e. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/0_overview/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/0_overview/ Here are RStudio guides for Workbench: a small one, and a big one. The problem is that unless you are an experienced devops engineer you may find some (a lot of) instructions unclear. Some of them may even be misleading. This set of pages is written to serve as a supplement to RStudio guides. It covers details relevant specifically to our way of setting up Workbench and simply things that might be unclear in general.