https://handbook.gradientmetrics.com/ Recent content in Gradient Handbook on Handbook Hugo -- gohugo.io https://handbook.gradientmetrics.com/operations/people-policies/general-policies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/general-policies/ Schedule Professional Development Schedule Gradient does not have official work hours, principally because we haven’t needed to dictate a set schedule and our team is distributed across too many time zones to make that realistic. The expectation is that everyone attends the sprint planning, standup, kaizen, and other firm-wide meetings. Other than that, your schedule is yours to manage and communicate. Professional Development Although we don’t have a formalized professional development program, that does not mean professional development does not occur. https://handbook.gradientmetrics.com/marketing/mission-vision-pillars/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/marketing/mission-vision-pillars/ Mission We founded Gradient to be the research company we wanted to see in the world. We dedicate ourselves to being a consultative research partner for our clients and to bringing objective clarity to our partners' most important strategy challenges by pushing methodological boundaries. Vision We want to be a premium research brand sought after for our methodological prowess by every global top tier organization. Our clients will recognize and choose Gradient for our sophisticated methodologies and for the bespoke, consultative, and personal partnership we bring to each engagement. https://handbook.gradientmetrics.com/announcements/2026-announcements/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/announcements/2026-announcements/ 17 July: Vendor Keep/Drop Files ↕ We have formalized our policy for providing final keep/drop files to vendors. Our standard policy is to provide sample vendors with a single, final reconciled keep/drop file after field closes and final data cleaning is complete. This ensures we do not pay for responses that will ultimately not be used in the analysis. Please take a few minutes to review the policy. https://handbook.gradientmetrics.com/gradient/onboarding/90-day-play/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/onboarding/90-day-play/ Purpose & Expectations Instructions Purpose & Expectations The 90 Day Plan is to be used by New Hires and their managers as a tool to outline training goals within the first 90-days of employment. This document relates to the training issues assigned in Jira, and provides added context around how we break down your first 90 days into stages with distinct focuses. https://handbook.gradientmetrics.com/gradient/about-gradient/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/about-gradient/ Where did Gradient come from? By Tom Vladeck The short version of the story is that I’ve always had an interest in math — I was a math major at Pomona, got a master’s degree from the London School of Economics, and studied statistics at Wharton while getting my MBA. And during my MBA, I decided that I really wanted to “do” cutting-edge statistics full time, and that to do it the way I wanted to, I had no choice but to start a company and get clients on my own. https://handbook.gradientmetrics.com/ai-usage/ai-usage-guidelines/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/ai-usage/ai-usage-guidelines/ Purpose & Philosophy Purpose Core Philosophy Ethical Standards & Accountability Gradient’s Ethical Standards for AI Usage Accountability How We Use AI at Gradient Survey & Synthesis Analysis Operations Research-Specific Guardrails What To Watch For Human Review Standard Tone & Formatting Alignment Gradient Voice and Tone Avoid Clearly AI-Generated Formatting Voice & Tone Skill Data Privacy & Security Ongoing Evolution Additional reference: AI Use within Gradient’s IT Policy https://handbook.gradientmetrics.com/projects-clients/approaching-contacts/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/approaching-contacts/ Intermediary Collaborator Random Person Vendor Potential Client Existing Client Competitor The 50/50 Call Intermediary These are companies or people that would not necessarily use Gradient directly, but would have an interest in having us work with a connection of theirs to create value.For example: investment firms that have portfolio companies that would benefit from Gradient’s services. Potential Objectives: Introduce Gradient Get introductions to their contacts Potential Approaches: https://handbook.gradientmetrics.com/marketing/technical-marketing/cloudfare-domain-management/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/marketing/technical-marketing/cloudfare-domain-management/ Our domain (gradientmetrics.com) is hosted and managed on a platform called Cloudfare. You can use this platform to buy new domains or add subdomains to our already purchased domains. Generally, we won’t be buying new domains too often, but we are adding subdomains quite frequently. What is a Subdomain? In the Domain Name System (DNS) hierarchy, a subdomain is a domain that is a part of another (main) domain. For example, if a domain offered an online store as part of their website example. https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/code-testing-protocol/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/code-testing-protocol/ What makes code close to perfect? We test our code to assure it is safe What do we use to test our code? What do we use to verify whether code is properly documented? When is our code tested sufficiently? Overall recommendations What makes code close to perfect? Any code that we write should fulfill four requirements: It should be safe. It should be efficient. https://handbook.gradientmetrics.com/product-team/coding/coding-at-gradient/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-at-gradient/ Introduction The Gradient monorepo Setting up your Environment Logging in to RStudio Workbench Updating your .Renviron Using Git in our monorepo Git credentials Branches Pull Requests Introduction At Gradient we live and breathe R. We work with R every single day, and so it only makes sense to develop tools and workflows that help us code as easily and efficiently as possible. https://handbook.gradientmetrics.com/product-team/coding/coding-standards/coding-principles/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/coding-principles/ Gradient Coders 1. Keep it simple & stupid Over-complicating code is the bane of all coding issues. In the words of Antoine de Saint-Exupery, “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” 2. Keep it DRY (Don’t Repeat Yourself) Whenever you copy and paste a piece of code - there is almost always an easier solution (e.g. write a function). https://handbook.gradientmetrics.com/products/conjoint-experiments/what-is-conjoint/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/what-is-conjoint/ Introduction Varieties of Conjoint Analyses CBC (Choice-Based Conjoint) ACBC (Adaptive Choice-Based Conjoint) MBC (Menu-Based Choice) Key Conjoint Analysis Terms How Many Attrbutes, Levels, Concepts, & Sets are Ideal? How Many Responses are Needed? Conjoint FAQ Who invented conjoint analysis? Is conjoint analysis qualitative or quantitative? Why is conjoint analysis used? What is choice-based analysis? What is adaptive conjoint analysis? What is discrete choice analysis? https://handbook.gradientmetrics.com/operations/data-security/security-tools-policies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/data-security/security-tools-policies/ Introduction VPN: NordLayer Password Management: NordPass Antivirus: ESET Introduction To ensure the security of company and client information, as well as to comply with multiple Data Processing Agreements we have with clients, Gradient uses the following data security tools: NordLayer, NordPass, and ESET. The purpose of this page is to clarify the policies for each data security tool. Please follow these rules! https://handbook.gradientmetrics.com/projects-clients/deliverables/hall-of-fame/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/deliverables/hall-of-fame/ Gradient’s Client Deliverables HOF Segmentation Conjoint & Willingness-to-Pay Survey Experiment CLV / Appraise Occasion Analysis Qualitative Text Analysis Crunch Dashboard See the Google Suite Templates page for further details on how to locate and utilize templates for our deliverables—and why we use always use templates. Gradient’s Client Deliverables HOF Below is a list of some of the most exemplary client deliverables that Gradient has delivered. https://handbook.gradientmetrics.com/newsletter/developing-instrument/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/developing-instrument/ Introduction Timeline Deciding on a Module Topic Writing Modules Analysis Specification Introduction Developing the survey instrument is the first stage in the newsletter development cycle. Some stages of the newsletter development process require creativity, some are more programmatic/task-based, and some are analytical. This is perhaps the only stage that requires using all three of those skills in one set of tasks. https://handbook.gradientmetrics.com/tools/Dynata/api-guide/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/Dynata/api-guide/ Step 0: Credentials Step 1: Load packages Step 2: Set up project Step 3: Update survey Step 4: Set up Dynata project Step 5: Create the project Step 6: Test redirect URLs Step 7: Buy Dynata project Also covered on docs.gradientmetrics.com here Step 0: Credentials Make sure you have the following credentials specified in your .Renviron file DYNATA_API_HOST = https://api. https://handbook.gradientmetrics.com/operations/employment-law/equal-employment/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/employment-law/equal-employment/ It is the policy of the Company to provide equal employment opportunity for all applicants and employees. The Company does not unlawfully discriminate on the basis of race, color, creed, age, perceived age, national origin, alienage, ancestry, citizenship status, gender (including sexual harassment), gender identity and expression, sexual orientation, transgender status, gender dysphoria, physical or mental disability, medical condition, marital or familial status, partnership status, pregnancy and caregiver status, sexual or reproductive health decisions, immigration status, religion, political affiliation or belief, genetics, military or veteran status, sex, genetic predisposition and/or carrier status, prior arrest or conviction record, domestic violence victim status and unemployment status and any other basis protected under local, state or federal law. https://handbook.gradientmetrics.com/operations/finance/finance-day-checklist/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/finance-day-checklist/ Introduction Checklist Client Billing Vendor Invoices Quickbooks Cash Flow Frog Ramp (Finance Fridays only) Monthly Reconciliation Process Introduction This document serves as an overview of tasks done on “finance days,” also know as Money Monday, Wealth Wednesdays, and Finance Fridays. Checklist Client Billing Review open invoices in Quickbooks: Sales > Invoices > Status: Unpaid > look for statues that are “Sent” or “Viewed” Reconcile any Unbilled Income: Sales > All Sales > Unbilled income Match any new payments with invoices: Transactions > Bank Transactions> For Review: look at both Wells Fargo accounts Review, finalize, send client invoices due to be sent Reconcile with Opportunity Tracking Vendor Invoices Review vendor invoice submissions, submit new ones to bill. https://handbook.gradientmetrics.com/product-team/coding/other-advanced-topics/fonts/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/other-advanced-topics/fonts/ General fonts overview Font types Outline fonts Fonts in Ubuntu Standard Font Locations in Ubuntu Installing fonts on Ubuntu Fonts in R Registering fonts for pdf and PostScript with extrafonts 1. Importing Fonts 2. Registering Fonts Font embedding Font matching Maintaining a package with custom fonts General fonts overview Font types Fonts vary in structure and use, with each type having its own characteristics. https://handbook.gradientmetrics.com/tools/ghost-deck-app/ghost-deck-101/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/ghost-deck-app/ghost-deck-101/ Introduction Make a Plot in 5 Seconds Training Videos Ghost Deck App Session Introduction The Ghost Deck app is an in-house tool designed with the research managers in mind for creating plots that would be included in a ghost deck. Please note all values generated from this app are FAKE and should ONLY be used in the context of a ghost deck deliverable. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/1_good_to_know/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/1_good_to_know/ Connecting through ssh File Editors Disk Usage Effective Bulk File Transfers General things good to know before doing any Workbench management related tasks. Connecting through ssh Terminal command to connect to an instance: ssh -i <key pem file> ubuntu@<instance public IP v4 DNS> Example: ssh -i RStudio_Workbench.pem ubuntu@ec2-35-173-234-97.compute-1.amazonaws.com Instance public IP v4 DNS can be found at the instance page under the network tab. https://handbook.gradientmetrics.com/handbook/handbook-usage/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/handbook/handbook-usage/ Why Handbook First Documenting in the handbook before taking an action may require more time initially because you have to think about where to make the change, integrate it with the existing content, and then possibly add to or refactor the handbook to have a proper foundation. But, it saves time in the long run, and this communication is essential to our ability to continue scaling and adapting our organization. https://handbook.gradientmetrics.com/sales/sales-playbook/intro/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-playbook/intro/ The teamwide training on How to Tell the Gradient Story, conducted by Kyle on 29 August, is below. The deck can be found on the Drive here! https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/hygiene-of-directories-structure/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/hygiene-of-directories-structure/ Readme The current structure of our monorepo is listed in the root-level README.md file. In addition to that this files contains a summary of our github actions builds statuses. It is the responsibility of the monorepo maintainer to ensure that new additions to the monorepo are placed according to the structure; when necessary make changes to the structure; keep README.md up-to-date. Keep the source not the output This is general rule applied to everyone to avoid adding to git those files that can easily be generated using the necessary scripts. https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-jira-rm/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-jira-rm/ Introduction Roadmapping Releases Sprint Planning Issues for Contractors Assigning Review Issues Spec Writing Best Practices Spec’ing for Yourself Spec’ing for Other Team Members Providing Context Headlines Formatting Examples Tagging Team Members Introduction This documentation focuses specifically on the best Jira practices for research managers. For more general information on Jira hygiene and best practices, click here. https://handbook.gradientmetrics.com/tools/JIRA/user-management/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/user-management/ Adding New Users Adding Users to Groups Logging In As Another User Personal Dashboards This page is meant for JIRA Admins and presents the content of the User Management panel. Adding New Users Go to the User Management panel and invite users with the Invite users button. On the next screen you can pass a user’s e-mail and invite them to the jira-administrators group that allows access to all projects. https://handbook.gradientmetrics.com/products/list-experiments/what-is-list-experiments/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/list-experiments/what-is-list-experiments/ Introduction Illustrative Example Why a List Experiment? List Experiment Terms & Concepts Designing a List Experiment Step 1: Identify Sensitive Statement(s) To Be Tested Privately and Publicly. Step 2: Assemble Control Item Bundles. Examples Step 3: Design Survey Step 4: Determining Sample Size and Configuration Step 5: Randomly Assign Respondents To Control Or Treatment Groups Step 6: Collect Data Step 7: Analyze Results List Experiment FAQs Why is the estimate sometimes negative? https://handbook.gradientmetrics.com/products/market-sizing/what-is-market-sizing/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/market-sizing/what-is-market-sizing/ TAM, SAM, and SOM Overview Example of each phase for Oatly When to Propose a TAM, SAM, and/or SOM Market Evaluation Market Segmentation Competitive Analysis Resource Allocation Goal Setting Market Sizing in Action Examples Launching a new product Entering a new market For many of our clients, it is essential they understand the size of their market. https://handbook.gradientmetrics.com/products/maxdiff-experiments/what-is-maxdiff/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/maxdiff-experiments/what-is-maxdiff/ Introduction When to use MaxDiff? Designing MaxDiff Experiments Rules of Thumb Design with large numbers of alternatives Prohibitions Anchored MaxDiff Introduction Our customers are constantly making choices, some conscious, but most unconscious. Regular surveys questions with traditional question scales are unable to get to the core of why a consumer makes his or her decision. Most of these scales don’t simulate real-life conditions, which always include some type of real trade-offs. https://handbook.gradientmetrics.com/gradient/performance-review/perform-eval-process/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/performance-review/perform-eval-process/ Introduction Continuous Support Principled Compensation Changes Process Process Schedule Dimensions & Rubric Rating Scale Evaluation Survey Bonus Allocation Three Prongs Formula Individual Performance Relative to Firm Average Merit-Based Raises FAQs Introduction Most companies have some form of Performance Evaluation Process, and Gradient is no different! However, Gradient has created a process that is thoughtful, intentional, streamlined, and hopefully without the “corporate” feel of other companies’ processes. https://handbook.gradientmetrics.com/sales/sales-introduction/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-introduction/ The Role of a B2B Salesperson At Gradient Metrics, you are far more than a Research Manager. You are trusted advisors to our clients. You are storytellers, and problem-solvers. Your ability to dig into our customer’s goals, and whatever is getting in the way of them reaching their goals in a consultative way, is part of our competitive edge. It is your attention to detail that makes those relationships great. So, in many ways, you are already practicing much of what it takes to bring new business into our organization. https://handbook.gradientmetrics.com/products/segmentations/what-is-segmentations/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/segmentations/what-is-segmentations/ Introduction Segmentation Analysis at Gradient Key Segmentation Analysis Terms How many segments are ideal? How many respondents are needed for a segmentation analysis? Segmentation FAQs Why is Segmenatation analysis used? What is a Typing Tool? What does Index, segment proportion and population proportion means? What are the steps in segmentation analysis? Introduction Segmentation is the process of splitting a large business market into sub-groups based on some form of shared characteristics. https://handbook.gradientmetrics.com/tools/slack/notifications/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/slack/notifications/ This page is meant for ??? and presents the content of the ???. Writing to Slack from R This document is based on Slack’s Documentation regarding Incoming Webhooks. Below is the Step by Step guide Create a Slack app Hit this link Select Incoming Hooks Activate Incoming Hooks Add New Webhook to Workspace and pick a channel Use webhook in R code The webhook will appear, like this; https://handbook.gradientmetrics.com/sales/sales-process/stage-lead/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-process/stage-lead/ The Gradient Client Journey Get/Who/To/By The Gradient Client Journey Matrix Definition A Lead is an individual or organization that has shown interest in Gradient Metric’s products and services, but is not yet qualified to buy. The Lead Stage is the initial phase in the sales process where potential customers (Leads) are identified. These are individuals or businesses that should ideally mirror our ICP attributes. https://handbook.gradientmetrics.com/research-team/statistics/statistical-rethinking/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/statistics/statistical-rethinking/ In 2020, Gradient wanted to collectively up their statistical knowledge. We did this using Richard McElreath’s Statistical Rethinking as our starting point. If you’d like to order your personal copy of Statistical Rethinking. Feel free to use your RAMP card. Contents Overview Chapter list & Team Videos Overview Statistical Rethinking: A Bayesian Course with Examples in R and Stan builds your knowledge of and confidence in making inferences from data. https://handbook.gradientmetrics.com/handbook/handbook-style-guide/style-basics/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/handbook/handbook-style-guide/style-basics/ Font Headings Capitalization Use Title Case for Titles and Headings Capitalize Brand Names Use Kebab Case for File and Directory Names Links Link Guidelines URLs URL Guidelines Understanding Absolute and Relative URLs Adding Images & Videos Uploading Images To Hubspot Font Gradient uses IBM Plex Sans (sans serif) font in most cases for both internal and client-facing deliverables. https://handbook.gradientmetrics.com/research-team/survey/survey-workflow/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey-workflow/ Purpose Workflow at a Glance Survey: Writing Structural Setup and Internal Alignment Survey Outlines What Makes a Strong Survey Outline Moving From Outline to Full Survey Sample Parameters Variable Naming and Programming Instructions Review, Client Approval, and Survey Lock Survey: Programming Survey: Soft Launch Survey: Data Collection Survey: Data Cleaning Vendor Keep/Drop Files Standard Policy Why We Do Not Routinely Provide Interim Keep/Drop Files Longer Field Periods and Vendor Payout Cycles Exceptions Boilerplate Language For Vendor Requests Purpose This page documents Gradient’s standard, end-to-end workflow for survey projects, from initial research question through to final, analysis-ready data. https://handbook.gradientmetrics.com/products/misc-analyses/text-analysis/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/misc-analyses/text-analysis/ https://handbook.gradientmetrics.com/research-team/survey/qualtrics-programming/translating-surveys/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/qualtrics-programming/translating-surveys/ Using Qualtrics Interface to Translate Surveys How to Preview Your Translated Surveys Limitations of Qualtrics Translation Interface Conjoints Using Qualtrics Interface to Translate Surveys For many clients, we need to translate our Qualtrics surveys into multiple languages. You can create and edit translations by going to the Translations tab in the survey builder. Next, select “Add language,” and click on the languages you will be translating the survey into. https://handbook.gradientmetrics.com/gradient/onboarding/type-of-work/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/onboarding/type-of-work/ (Very small) Sample of Products Newsletter Gradient’s Newsletter is free for our subscribers. We use it as a vehicle to showcase our original research and attract new business opportunities. We field our own surveys to create datasets that we use for new editions of the Newsletter. Segmentation Audience segmentations are where we have the deepest experience. We’ve done segmentations of voters, the general public, and of many different specific consumer categories (e. https://handbook.gradientmetrics.com/sales/sales-training/what-to-expect/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-training/what-to-expect/ Introduction Objective The Role of a B2B Salesperson What You Can Expect to Learn over the Coming Training Sessions Key Definitions* The Baku Approach to Training In Closing Introduction We are Peter Kuhn and Guy Barbier from Baku. We support B2B companies on how to find, and maintain their go-to-market footing. We’ve spent the last 6 months in the weeds with Gradient Metrics, helping Kyle start new conversations, advance existing conversations, and bring on new partners. https://handbook.gradientmetrics.com/product-team/devops/APIs/working-with-r-apis/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/APIs/working-with-r-apis/ Useful resources How to prepare an EC2 instance to host APIs made in R 1. Run an ec2 2. Connect to the machine through ssh. 3. Redirect rstudio to port 2000 4. (optional) Assign an elastic IP to the instance 5. Login into RStudio on port 2000 6. Prepare plumber code 7. Launch the API Examples Typing Tool called by Qualtrics Example for client project: Chalk Setup Qualtrics Restarting the EC2 machine API monitoring Useful resources REST APIs and Plumber on R Views. https://handbook.gradientmetrics.com/product-team/devops/aws/working-with-r-on-aws-ec2/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/working-with-r-on-aws-ec2/ Basics Overall guidelines How to run an EC2 box How to start a new R project? How to work with an ongoing project? Using renv Turning off the machine for the night Shutting down the machine completely More Basics What is Amazon EC2? StackOverflow thread Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. https://handbook.gradientmetrics.com/gradient/company-guidelines/working-with-tom/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/working-with-tom/ How to use this document: Use this document to figure out when and whether you should get in touch with me, how you should do so (depending on the task at hand), what you should put in front of me, and how to be impressive in general. The kinds of things I like to get involved in: By “get involved” I mean things where I am actively contributing work or direction. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/conjoint-types/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/conjoint-types/ Introduction Full-Profile (AKA Basic CBC Design) Partial-Profile Alternative Specific Design The “None” Alternative" Introduction As mentioned in the previous chapter, conjoint studies come in many flavors. The number of attributes or even the type of attributes (discrete vs. continuous) between each conjoint can change. Some conjoints have prohibitions (e.g., not showing Blue and Ferrari as levels in one profile), and sometimes the sheer number of attributes / levels is just too much to show. https://handbook.gradientmetrics.com/sales/ideal-customer-profile/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/ideal-customer-profile/ ICP Definition General Guidelines Blue chip and legacy brands Creative and brand agencies Philanthropies, think tanks, and political organizations ICP Definition Gradient serves three distinct types of clients that fall into larger “buckets”: Blue chip and legacy brands Creative and brand agencies Philanthropies, think tanks, and political organizations There are ideal customer profiles (ICPs) among those types of clients. While many of the roles and demographics are shared across the client “buckets,” their pain points and goals demand discrete buying personas. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/introduction/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/introduction/ TL;DR DisplayR Design Module Overview TL;DR If you feel smart, and don’t want to waste time on brushing up your conjoint knowledge, you can go directly to the short tutorial on the DisplayR settings: Otherwise continue reading. Each conjoint study has a few critical parts that contribute to a successful project. We are capable of controlling some of these parts, while others are kept random. https://handbook.gradientmetrics.com/gradient/team/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/team/ Put names to faces. Use this directory to find out more about the Gradient team, including but not limited to a person’s timezone, preferred work schedule, address, chakra’s and ocean profile. Thomas Vladeck Founding Partner Thomas Vladeck Founding Partner Brooklyn, New York 09:00 - 17:00 ET +1 (202) 390-0838 Calendly How to work with me Kyle Block Chief Executive Officer Kyle Block Chief Executive Officer Amsterdam, The Netherlands 08:30 - 17:00 GMT+1 +1 (213) 361-7875 Stefan Musch https://handbook.gradientmetrics.com/research-team/survey/survey-panels/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey-panels/ Tl;dr Checklist Introduction Step-by-Step Guide Step 1: Acquire feasibility, cost, and timeline estimates Desired sample parameters Description of target population and targeting specs Desired N Quotas Length of interview Timeline Request the following… Panel contact information Step 2: Update Qualtrics survey Important embedded data fields Request & update redirect links Update quotas Tracking qualified completes Step 3: Test survey links Step 4: Share the quota dashboard Step 5: Approve soft launch Step 6: Approve full launch & monitor data collection Tl;dr Checklist Acquire feasibility, cost, and timeline estimates Update Qualtrics survey Embedded data fields Redirect links Quotas Test survey links Share quota dashboard Approve soft launch Approve full launch & monitor data collection Introduction Are you fielding a survey through a panel? https://handbook.gradientmetrics.com/gradient/company-guidelines/spec-writing/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/spec-writing/ Why Do We Spec? Remote Collaboration Analogies Galore! Spec’ing Is a Sign of Brilliance, Not Weakness What to Include in a Spec 1. Context: Set the stage 2. Deliverables: Identify the Output 3. Plan: Outline the Steps to Get There Examples of Spec Features by Type When to Spec You DO Need a Spec When: You Probably DON’T Need a Spec When: Best Practices and Examples Spec’ing Mondays Be You! https://handbook.gradientmetrics.com/operations/documentation/google-drive/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/documentation/google-drive/ Google Drive File Structure Google Drive is where most of our deliverables, summaries, proposals and additional content live. All client and project documentation must be stored within a Shared Drive. The most important part here is to create meaningful names. When reading a file name, everyone should instantly know what purpose it serves. Additionally it is extremely important to save your files in the proper location. Find out if there are folders associated with your type of content, or create additional folders to avoid clutter. https://handbook.gradientmetrics.com/operations/documentation/gsuite_templates/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/documentation/gsuite_templates/ Why Use a Template Accessing Templates Google Slide Themes Adjusting Templates Whether you are creating a survey document, memo, ghost deck, or more, we have likely already built a Template for that exact deliverable type. Utilizing these Templates is very advantageous and should always be the priority over starting a blank document. Our Templates are saved in the Google Shared Drives and accessible to all. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/algorithms/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/algorithms/ Algorithms Complete Enumeration Balanced Overlap Efficient Partial Profiles Alternative specific Random and Shortcut Algorithms The first thing you need to do is to decide which algorithm to use. The options are: Random Shortcut Complete Enumeration Balanced Overlap Efficient Partial Profiles Alternative specific - Random Alternative specific - Fedorov If you are puzzled, don’t worry, this is not as scary as it looks. https://handbook.gradientmetrics.com/product-team/devops/APIs/overquotas-api/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/APIs/overquotas-api/ The video Purpose Overview The code Setup in Qualtrics The video This page is a result of this JIRA card during which we created the below video. Purpose The main motivation for this service is that Qualtrics charges us for data collection when respondents are marked as “overquota”. An overequota respondent means that a respondent is screened out (not eligible) of the survey. https://handbook.gradientmetrics.com/projects-clients/deliverables/best-practices/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/deliverables/best-practices/ BLUF: Bottom Line Up Front Headlines Should Describe the Key Finding Use Icons to Concisely Summarize Lengthy Group Descriptions Use Colors to Your Advantage Use Consistent Color Palettes, Icons, and Patterns of Conveying Information* Use a Tutorial or “Primer” Slide to Help Your Audience Interpret Complex Tables or Plots Use Annotations Liberally to call Out Secondary Findings Use Visual Plots to Convey Magnitude Differences Use Consistent Slide Appearances Regardless of what deck (or any deck at all) you use for inspiration, below are a few guiding principles that researchers should keep in mind when designing client deliverables. https://handbook.gradientmetrics.com/products/misc-analyses/clv-analysis/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/misc-analyses/clv-analysis/ https://handbook.gradientmetrics.com/product-team/coding/coding-standards/coding-hygiene/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/coding-hygiene/ Fewer models, fewer variables We do not filter results until the very end Use saved / intermediate results sparingly Keep track of observations Everything should be contributing to a deliverable Code formatting Fewer models, fewer variables One data frame for each unit of analysis. E.g. Customer Transaction Item One model for each considered dependent variable and features' set Use the model that performs and suits the best Leave only the model with the features' space that was finally used Use longer pipes to avoid partial data. https://handbook.gradientmetrics.com/gradient/onboarding/communication/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/onboarding/communication/ Communication Tools Google Workspace Slack Zoom Our communication hierarchy Communication Tools During your first few days, you’ll want to become acquainted with all of the following communication and collaboration tools. Some you will use more than others, but you should have a foundational understanding of what each is used for. Google Workspace We use Gmail for client and external communications, not for internal communication. https://handbook.gradientmetrics.com/operations/people-policies/compensation/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/compensation/ Principles Merit-Based Raises Bonus Allocation Commission Types and Rates Payment Principles Gradient’s goal is to pay at or above the market rate for each employee Each employee has a different market rate, determined by their skillset, region, and experience Compensation is never considered on a relative basis (what others at Gradient are making) Where possible, we try to link a substantial portion of compensation to performance in ways that can be measured objectively We review compensation every year Merit-Based Raises For employees that are eligible, merit-based raises are determined by: https://handbook.gradientmetrics.com/products/conjoint-experiments/conjoint-tutorial/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/conjoint-tutorial/ Intro Planning Design Design API Design Checks Qualtrics Programming Step 1: Create Embedded Data Fields Step 2: Import Design File Step 3: Program Loop & Merge Step 4: Program the Choice Task Checking with MTurk and Soft Launch Modeling Analyses Plotting Synthesis FAQ Intro Every Conjoint analysis consists of several steps. To assure that the projects are going smoothly, we should be aware of the crucial parts of each step. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/conjoint-vocabulary/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/conjoint-vocabulary/ Vocabulary Since people love to complicate simple things, there are a few conjoint related terms that have a bunch of synonyms that might confuse you. Below is a list of possible synonyms (in brackets) for conjoint terms (the ones we should use are in bold) with a short description. Version [block]: a set of conjoint tasks that will be presented to a single respondent. The tasks within the version are optimized in a way that the combinations of attribute levels are unique and balanced (the levels of each attribute appear a more or less equal number of times). https://handbook.gradientmetrics.com/product-team/coding/package-development/messages/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/messages/ Errors and warnings Alerts Inline text formatting Collapsing inline vectors Line breaks Headings Lists Bigger text paragraphs or code chunks Progress Bars Requesting user input Other things to be aware of Examples Let’s make console messages in our packages pretty and unified! Base R tools for this are quite limited, but luckily there’s a package called cli with a wide range of possibilities. https://handbook.gradientmetrics.com/operations/data-security/security-tools-setup/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/data-security/security-tools-setup/ Introduction VPN: NordLayer Password Management: NordPass Exporting from the LastPass browser extension Exporting from the LastPass website Importing passwords using the desktop application and Web Vault Antivirus: ESET Purpose Endpoint Onboarding 1. Send the Installer and Instructions Instructions for Employee 2. Confirm the Endpoint Appears in ESET PROTECT 3. Confirm Required Modules & Subscriptions 4. Review Alerts 5. Update the Endpoint Inventory Endpoint Offboarding 1. https://handbook.gradientmetrics.com/product-team/coding/other-advanced-topics/shinyapps/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/other-advanced-topics/shinyapps/ Setting Up rsconnect Deploying: the simple case Deploying with private packages Known issues 1 2 3 4 Setting Up rsconnect To deploy your Shiny app on shinyapps.io, first ensure you have an rsconnect account set up. Check existing accounts in RStudio using rsconnect::accounts(). If you need to set up a new account, visit the ShinyApps Dashboard. Here, you’ll find a pre-filled rsconnect::setAccountInfo() command. https://handbook.gradientmetrics.com/handbook/handbook-editing/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/handbook/handbook-editing/ The Importance of our Handbook Why a Git Repository? How is the Handbook Built? Flow Structure Editing the Handbook Online Editing the Handbook Locally Building & Previewing the Handbook Locally Organizing the Handbook Content Folder Content/post Folder Creating a New Page Committing & Merging Your Changes Creating a Pull Request The Importance of our Handbook This handbook is our single source of truth and needs to be preserved that way. https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/editing-workflows/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/editing-workflows/ Proposed Workflow Best practices for updating current workflows or developing new workflows, the first step should be creating a diagram of the Proposed Workflow using Google Slides. Here are a couple examples of proposed workflow documentation: Proposed Updates to Survey Workflow Proposed Onboarding Workflow Creating a proposed workflow allows you to attach it to a Jira issue and assign it for review and feedback from other Gradient team members. https://handbook.gradientmetrics.com/gradient/performance-review/dimensions-and-rubric/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/performance-review/dimensions-and-rubric/ Context Context The Performance Evaluation Rubric document defines the dimensions along which all Gradient employees will be evaluated on a bi-annual basis. It also lays out the rubrics that all team members and their managers will use to evaluate performance based on the employee’s role: Research, Product, or Internal—please reference your respective rubric in the document. The results from the evaluation will then feed into the overall bonus allocation formula and implicitly inform the possibility and magnitude of a merit-based raise. https://handbook.gradientmetrics.com/operations/employment-law/harrassment-and-discrimination/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/employment-law/harrassment-and-discrimination/ Sexual Harassment Scope What Is Sexual Harassment? Examples of Sexual Harassment Complaint Procedure Supervisor Responsibilities Confidentiality of Complaints Responding to Complaints Retaliation What Is Retaliation? Agency Complaint Procedure and External Remedies New York State Human Rights Law (“HRL”) Civil Rights Act of 1964 Local Protections The Company is committed to treating all employees with respect and dignity and believes that its employees deserve a workplace that is safe and free from harassment or other discrimination. https://handbook.gradientmetrics.com/product-team/devops/aws/hosting-websites-on-s3/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/hosting-websites-on-s3/ This page is a summary of materials from github.com/MarcinKosinski/RonAWS What is S3? S3 bucket as a website Enabling static website hosting Website protection Protecting an S3-hosted website Adding a new user to all protect sited Setting up Cognito What is S3? Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. https://handbook.gradientmetrics.com/sales/sales-playbook/discovery/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-playbook/discovery/ During the discovery call: Opening: whether you’re talking to the same audience from a previous conversation or there are new stakeholders involved in the conversation, make sure to set the stage on the state of state before kicking off the call. The goal is to set the context for discussion and remind/reorient the audience on how we got to where we are today, what we’ve learned through our conversations to date, recap on initiative + pain points the organization is looking to solve, etc. https://handbook.gradientmetrics.com/gradient/company-guidelines/speed/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/speed/ Imagine you run a grocery store, and you are deciding which product to stock on your last unit of shelf space; although they are different products, they are exactly the same in almost all respects: they cost the same to purchase wholesale (let’s say $1 each), they are the same size and weight, etc., and they have no interaction with any other products — but you can sell the first product for $5 but the second for only $3. https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-survey-workflows/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-survey-workflows/ Introduction Survey Outline What they are and when to use them What to include in a survey outline Full Surveys Sample parameters grid Variable naming conventions Adding programming instructions Display logic In-question instructions Embedded data Common programming instructions Setting up a data provider, programmer, and reviewer Data provider Programmer Reviewer Launching Surveys Data Cleaning Submitting Data Invoice Introduction The purpose of this documentation is to introduce Gradient’s process from research question to final collected data, including processes around survey design, programming, data collection, and other related tools and steps that Gradient uses to collect high-quality data. https://handbook.gradientmetrics.com/products/market-sizing/methods-measuring-tam/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/market-sizing/methods-measuring-tam/ Top-Down Approach Bottom-Up Approach Value Theory Approach The Market Share Approach There are four broad methods of measuring TAM. As a company, we are most prepared to use the Bottom-Up approach. For that reason, I only briefly explain the other three methods (for more information on the pros and cons of each method, see a few useful resources here): Top-Down Approach This method estimates TAM by using industry reports and studies to calculate the total market size (for examples, see here and here). https://handbook.gradientmetrics.com/operations/finance/month-end-close/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/month-end-close/ Introduction Process Review P&L Report Review/Complete Q&A Worksheet Budget vs. Actual Report Final Reconciliation Introduction This document serves as information on the Month End Close process. Process Email is received monthly, usually (but not always) around the 3rd Thursday of the month, with a P&L Report and Q&A Sheet attached. Recurring Jira issue for month end close, is created every third Sunday of the month (assigned to Amy). https://handbook.gradientmetrics.com/gradient/our-principles/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/our-principles/ Gradient Leaders…* 1. Are honest and integral to a fault: If we say something will be done, then it will be done. We conform our words to reality (honesty) and reality to our words (integrity). 2. Do more with less: We are frugal and look for ways to avoid spending time and money when it is not needed. 3. Think win-win: Success is not zero-sum. Our clients, partners, and vendors’ success is our success. https://handbook.gradientmetrics.com/newsletter/programming-survey/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/programming-survey/ Introduction Timeline Programming Practices Specific to Trendlines Introduction After finalizing the instrument, the next step is to program the survey in Qualtrics. This section is brief because survey programming is to be documented elsewhere in the Handbook. Timeline This step comes right after developing the survey instrument and is part of the first phase of the newsletter cycle, which usually begins during the last week of the month. https://handbook.gradientmetrics.com/projects-clients/proposal-guidelines/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/proposal-guidelines/ Capture and Deliver what the Client Needs Pricing Approach Preparing a Budget Set Stage for Subsequent Business Review and Approval Capture and Deliver what the Client Needs Resist the temptation to offer a potential client more than what they need or ask for. In the long-term, Gradient will secure more business by excelling at what our clients demand of us than by jeopardizing a relationship by over-selling. https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/pr-review/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/pr-review/ You were asked to review a PR? Here’s what you should keep in mind: General approach By default we don’t do in-depth review of everything unless it was specifically requested. Sometimes PRs are just too big to review everything, and more often than not it would’ve been a waste of time. So it’s a responsibility of the person who asks for a review to clearly specify what needs to be carefully reviewed, unless the scope of the PR is very limited and the subject of review is crystal clear. https://handbook.gradientmetrics.com/tools/JIRA/jira-bulk-upload/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-bulk-upload/ Overview Bulk Import Steps 1. Export Your Roadmap CSV 2. Prepare the Configuration File 3. Import Roadmap Into Jira 4. Verify The Issues Were Created Overview As outlined in the Roadmapping section, We use a Jira Roadmap Template to plan out project issues that will then be created in the respective Jira project. Using the Import work items from CSV tool in Jira, we can bulk import all of the roadmap issues in one action, removing the need to manually create each issue. https://handbook.gradientmetrics.com/sales/sales-process/stage-qualification/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-process/stage-qualification/ Pre-call (How we prep for this stage) What are we trying to learn? What are we trying to show? What does great look like (what constitutes success)? How to follow up Definition This stage is where we begin our sales pipeline. Once a lead has expressed any signs of interest in Gradients' offerings, or has mentioned, even if just vaguely, some sort of opportunity that would be a good fit for Gradient, we enter the qualification stage. https://handbook.gradientmetrics.com/research-team/statistics/stat-videos/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/statistics/stat-videos/ This page holds a collection of videosor tutorials related to statistical concepts. It’s gathered throughout the years whenever one of our team members needed to explain how a model worked, or to have clients understand a particular statistical methodology. Statistical Relationship Between Count & Timing Distributions Statistical Relationship Between Count & Timing Distributions https://handbook.gradientmetrics.com/research-team/survey/survey_rules/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey_rules/ Introduction TL;DR - Example Naming Rules Generic Module Variables Product Variables Conjoint MaxDiff Segmentation Brand Tracker List Experiment Other naming conventions & rules: Multiple products in a single study Variable Naming Rules Prefixing Module Names Naming Variables Explicitly No Trailing Numbers Open-ended responses Introduction Gradient approaches survey writing a little different than most research companies. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2_system_configuration/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2_system_configuration/ Hardware EC2 EC2 role EC2 reservation FSx Lustre setup Creating a file system Installing the Lustre Client Mounting the file system RStudio Workbench installation Network settings Mapping web domain to an elastic IP https Application Load Balancer Network Load Balancer Overview Hardware Currently our Workbench setup uses one EC2 server with a mounted FSx Lustre storage. https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-strategies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-strategies/ Why write automated tests? Code testing strategies Example of tests Test inputs and outputs Errors and warnings Testing structure Why write automated tests? Testing code can be performed in two different ways: manual and automated. The first approach means that every time we write a new function, or refactor code, or add some new arguments to a function call, we would need to figure out what behaviour we expect. https://handbook.gradientmetrics.com/sales/sales-training/how-to-start-conversations/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-training/how-to-start-conversations/ Training Video Introduction Objective Key Topics Areas How much time should I focus on this? What types of customers are we pursuing? What is the right way to start a conversation with these people? What medium should I use to contact these people? What message should I use to contact these people? When is the right moment to ask for a referral? Are there etiquette guidelines when starting new conversations? https://handbook.gradientmetrics.com/product-team/devops/Automation/gh-action-with-service-account/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Automation/gh-action-with-service-account/ Why Use Service Accounts? Setting Up a Service Account Using the Token for Automation Tasks Important notes Example Implementation This guide explains how to automate R scripts using service accounts for non-interactive Google Drive authentication in GitHub Actions. This approach helps you bypass manual authentication prompts and enables seamless execution of tasks like updating Google Sheets or creating plots. Before we begin, this guide assumes familiarity with R scripting and basic GitHub Actions concepts. https://handbook.gradientmetrics.com/product-team/coding/working-on-projects/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/working-on-projects/ g6tr package Starting a new session Project structure Setting up and working on projects Sharing projects Reading and writing files to the cloud (GDrive and S3) Ending a project Please refer to the video below to accompany the written tutorial on this page. g6tr package First of all, if you are about to work on a project locally on your computer, make sure you have the latest version of the g6tr package, because it contains all our helper functions for managing projects. https://handbook.gradientmetrics.com/products/conjoint-experiments/analyzing-conjoint/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/analyzing-conjoint/ Introduction Conjoint Estimation Methodologies Aggregate logit Latent Class CBC Hierarchical Bayes Further Metholody Learning Conjoint Analysis Workflow at Gradient Using g6tr.cjt Reading conjoint data Asserting the data Tidying the data Modeling the data Extracting model statistics Introduction After finishing data collection, the data needs to be modeled and subsequently analyzed. While there are many ways (e. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2-1_system_configuration_disregarded/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/2-1_system_configuration_disregarded/ This page provides details about deprecated system configuration options. While we no longer use them, this information might come in handy in the future. Multi-attach EBS Installation Auto-Enabled IO Backups EFS Multi-attach EBS Installation Here’s the guide on creating a multi-attach compatible EBS volume: link. Select the io2 type. For size I chose 500 Gb, for IOPS - 1000. Size can be scaled up in the future (not down) and IOPS can be increased or decreased. https://handbook.gradientmetrics.com/gradient/Gradient-way-bible/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/Gradient-way-bible/ The purpose of this document is to lay out, to the best of our ability, the Gradient Way to working collaboratively. The Gradient Way is both a mindset and a specific orientation to how one approaches any task. We like to think that our success is due to everyone striving to adopt the Gradient Way, which—yes—sounds a tad cultish. But like any good cult will tell you, a shared paradigm is kinda essential to achieving shared objectives. https://handbook.gradientmetrics.com/gradient/communication/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/communication/ Types of communication Communication Hierarchy Working hours Managing Notifications Adjust Google Groups subscription Remove double notifications Notifying others in Slack Turn your mobile applications off after working hours Working remotely and communicating asynchronously have a ton of upsides. However, it can have many downsides too. And at Gradient we try to actively avoid these downsides in order to maintain a healthy work-life balance for everyone. https://handbook.gradientmetrics.com/product-team/coding/package-development/r-packages/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/r-packages/ Adding new R packages to our repository This has been automated with the g6tr_package_setup() function. But please recheck all the points manually. Create a new package inside the packages directory. Package folder should contain: R, tests, vignettes, data (optional), inst (optional), pkgdown folders DESCRIPTION, NEWS.md, .Rbuildignore files .Rprofile file. This should be the same as for all other packages. pkgdown/_pkgdown.yml file where functions and vignettes will be listed Nice to have: README. https://handbook.gradientmetrics.com/products/list-experiments/analyzing-list-experiments/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/list-experiments/analyzing-list-experiments/ Introduction Data structure Method Bayesian latent-opinion model (what we use) Other methods (briefly) g6tr.lx workflow 1) Parse survey data: lx_parse_data() 2) Fit the model: lx_fit() 3) Define subgroups: cohorts 4) Summaries and plots What the prediction data looks like Introduction A list experiment (also called the item count technique) is a survey experiment designed to estimate the share of people who agree with a sensitive statement when a direct question may be biased by social desirability, fear of judgment, or preference falsification. https://handbook.gradientmetrics.com/products/maxdiff-experiments/analyzing-maxdiff/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/maxdiff-experiments/analyzing-maxdiff/ Introduction Analyzing MaxDiffs with g6tr.md Using g6tr.md Reading MaxDiff data Asserting the data Tidying the data Modeling the data Extracting model statistics Alternative Analyses Count Analysis Hierarchical Bayes Latent Class Analysis Standard Logit Models Introduction A general guide on how MaxDiff works A practical example in R An in-depth case study A guide to MaxDiff in R with theMaxDiffpackage Analyzing MaxDiffs with g6tr. https://handbook.gradientmetrics.com/products/segmentations/analyzing-segmentations/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/segmentations/analyzing-segmentations/ Introduction Segmentation Methodologies NMF Further Methodology Learning Segmentation Analysis Workflow at Gradient Using g6tr.analytics Read data Extra Segmentation components Preliminary Analysis Modeling Silhouette Comparison NMF Workflow Others Introduction Following the completion of data collection, the data scientist is handed the data for segmentation analysis along with a few requests for the deliverables' specifications. These requests may differ depending on the project, but the basic steps will remain the same. https://handbook.gradientmetrics.com/operations/finance/cashflow-frog/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/cashflow-frog/ Introduction Scenarios Adding Projections Passthrough Expenses Account Mapping Introduction This document serves as instructions on how to manage Gradient’s Cash Flow Frog account and forecast. Gradient has not been using the Planned vs. Actual feature thus far. Recommend looking into using it moving forward. More information. Scenarios Multiple scenarios can be created by clicking on the + icon at the top of the dashboard. https://handbook.gradientmetrics.com/gradient/onboarding/collaboration/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/onboarding/collaboration/ JIRA Github Google Drive AWS Dockerhub HubSpot Forsta Crunch Upwork JIRA 📃 Given the initial complexity of JIRA, you will have a specialized JIRA training during your onboarding at Gradient. JIRA is our project and product management tool. Although it is commonly used by software companies, we’ve found it to be pretty helpful at helping us keep track of, specify, and review work product in a highly systematized way. https://handbook.gradientmetrics.com/product-team/coding/coding-standards/commenting-protocol/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/commenting-protocol/ This page shows you how we comment our code and what the general protocol is. Each assigned function needs to have properly documented parameters, a title, one line of description and at least one example. It should follow roxygen2 syntax, and should be written in markdown. See the below Conjoint function for a good example of an overly detailed function (as all our functions should be!) #' @import dplyr #' @importFrom aws. https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-guardrails/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-guardrails/ https://handbook.gradientmetrics.com/newsletter/fielding-dynata/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/fielding-dynata/ Introduction Timeline Monitoring the Launch Introduction Step-by-step instructions for launching the survey through the Dynata API can be found here. Timeline This step comes right after programming the survey and is part of the first phase of the newsletter cycle, which usually begins during the last week of the month. During an ideal fielding week, the timeline is as follows (we will be covering the bolded portions in this section): https://handbook.gradientmetrics.com/product-team/coding/package-development/deprecating-functions/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/deprecating-functions/ Introduction Further reads Application Introduction We encounter circumstances when a function requires significant modifications that might disrupt the flow of the original function (e.g., changing argument names, adding new arguments, etc.) or it simply need a thorough re-branding. Still, we don’t want to lose the existing function. It is feasible to notify existing users of those functions about the updates made so that people do not have to wonder why a function breaks or no longer works. https://handbook.gradientmetrics.com/gradient/company-guidelines/high-quality-writing/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/high-quality-writing/ Introduction Writing Tips You can’t do it in one pass Take your reader on a journey Be specific Make the case behind the position, don’t just state the position Read carefully If you can’t get clear on what you want to say, make that the issue Some resources Introduction Writing well is important for at least three reasons: the process of writing, and writing well, helps you think more clearly and develop better, more complete ideas; clear and crisp written communication is the most efficient way to transmit these ideas to someone else, especially when the ideas are complex; finally, leaving behind a written document memorializes a line of thinking that will be available for future people (that future person being you, perhaps) to inspect. https://handbook.gradientmetrics.com/products/market-sizing/implement-tam-sam/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/market-sizing/implement-tam-sam/ Step-by-Step Process Step 1: Define the product/service Step 2: Define the market Step 3a: Develop sampling criteria Step 3b: Develop survey questions Survey Questions for TAM Demographic & Market Specifics Identifying Potential Buyers Frequency of Purchases Survey Questions for SAM Specific Purchase Behaviors Willingness-to-Pay Step 4: Administer the survey Using our typical survey methods, we can relatively easily calculate a TAM, and with some agreed-upon criteria with a client, a SAM. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/inputs/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/inputs/ Common options Algorithm specific options Random, Shortcut, Complete Enumeration, Balanced Overlap and Efficient algorithm input options Partial profile input options Alternative Specific input options Common options The plethora of possibilities might seem too much to handle, but the Random, Shortcut, Balanced Overlap, Complete Enumeration, and Efficient have the same input menu. Only Partial profiles and Alternative Specific designs are different. https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-analysis-environment/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-analysis-environment/ Overview of Gradient Analysis Environment Statistical Analysis Tool: R R Workbench & Working on R Projects Amazon Web Services (AWS) Amazon EC2 (Elastic Compute Cloud) Amazon S3 (Simple Storage Service) GitHub Tips for Working with the Product Team Schedule Pre-Mortem Analysis Meetings Maintain Frequent Communication Spec Analysis Requests in Detail Overview of Gradient Analysis Environment At Gradient, we leverage advanced tools and technologies to tackle the complex modeling tasks our projects demand. https://handbook.gradientmetrics.com/operations/data-security/it-policies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/data-security/it-policies/ Overview Goals Definitions Endpoint Policy Acceptable Use Policy Purpose Use of Endpoints Use of Personal Mobile Devices Use of Email Use of the Internet Monitoring AI Use Removable Storage Printing Password Policy Purpose Scope Policy Complexity Requirements Joiner and Leaver Policy Joiners Process Leavers Process Data Protection Approved Data Storage and Transfer Systems Remote Endpoints Data Classification VPN Usage Policy Malware/Antivirus Software Policy Personally Identifiable Information Policy Backup Policy Service-Lead Design Cloud Service Backups Endpoint Backups Social Networking Policy Scope Policy Overview This document serves as Gradient's comprehensive IT Policy, to be read and understood by all parties working with the Gradient team, either internally or externally. https://handbook.gradientmetrics.com/tools/JIRA/jira-hygiene/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-hygiene/ Introduction Projects/Boards Roadmapping Backlog Issues Issue Types Smart Checklists Specs Status Approvers Labels Priority Recurring Issues Review Issues Story Points & Sprint Planning Story Points Why Story Points are Important Estimated vs Actual Story Points Sprint Planning Weekly Planning Releases/Fix Versions Resources Introduction We LIVE in Jira! As such, let’s make sure we are using these best practices with creating issues and project management for cohesion and collaboration. https://handbook.gradientmetrics.com/products/misc-analyses/market-basket-analysis/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/misc-analyses/market-basket-analysis/ https://handbook.gradientmetrics.com/research-team/survey/population-benchmarks/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/population-benchmarks/ Introduction Interfaces to Explore the U.S. Census Data Resources for Additional Population Benchmarks Introduction When developing surveys, we often need to create quotas using population benchmarks. Although the U.S. Census Bureau has the most exhaustive repository of population benchmarks, the website can be difficult to use and, in some circumstances, does not contain the specific benchmarks we are looking for. Luckily, there are interfaces that make exploring census data easier and additional resources available to explore population benchmarks that are not in the census. https://handbook.gradientmetrics.com/projects-clients/proposals/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/proposals/ Guidelines Jira Issue Template Documents Workflow Terms Guidelines We typically do proposals via memo, not decks—this allows for more substantive conversation than a deck. Before telling them what we’ll do, we align with them on what they need. We don’t put pricing down until we’ve done a pricing analysis. In an ideal world, proposal development would be split into two stages: FIRST objective and methodological alignment, THEN pricing determination. https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/branch-review/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/branch-review/ On github page of our monorepo, go to branches -> All branches. In the perfect world you should only see gray or green PR icons and all PRs should not stay inactive for more than 2 months. Gray indicates a draft PR that is in progress. Green means a PR that is ready to be reviewed. The whole process of branch review is essentially about achieving this perfect picture. We have a recurring Jira issue which pops up every two weeks and assigns one person to do the branch review. https://handbook.gradientmetrics.com/product-team/coding/renv-issues/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/renv-issues/ Before renv::restore() Installing packages from GitHub Troubleshooting renv::restore() Suggested troubleshooting process Specific package installation issues Biobase Package cannot be loaded There’s no package called … Problematic GitHub installation record Other issues File access issue Before renv::restore() So far there’s just one thing worth calling out: Installing packages from GitHub When installing packages from GitHub, prefer using a commit SHA (not a PR ref such as e. https://handbook.gradientmetrics.com/sales/sales-process/stage-opportunity/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-process/stage-opportunity/ Pre-call (How we prep for this stage) What are we trying to learn? What are we trying to show? What does great look like (what constitutes success)? How to follow-up Definition This stage is where a lead has been qualified through the lens of ICP and BANT and the prospect is actively pursuing solutions to solve the problem within a reasonable time horizon (generally 60-120 days. https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-inside-function/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-inside-function/ Why test inside a function? It’s important to have tests that will serve as a sanity check inside our functions. These tests are usually made to throw an error if something is wrong with the entered parameters, which are passed to a function call. For instance, it can be valuable to check if an parameter is of character type or if all the necessary columns are present in a data.frame. https://handbook.gradientmetrics.com/product-team/devops/aws/user-management/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/user-management/ Adding a new user to AWS Creating a policy to work with a specific directory within an S3 bucket Adding a policy to an existing user Adding a new user to AWS Log in to the console https://console.aws.amazon.com/console/home?region=us-east-1 Go to IAM service https://console.aws.amazon.com/iam/ Go to Users https://console.aws.amazon.com/iam/home#/users Click Add user Provide a user name. On the access type pick AWS Management Console access and specify a password for the user Permissions Admin: On the next tab, that provides permissions select Copy permissions from existing user and pick mkosinski or smusch user if you want this user to be an Admin External programmer: On the next tab, that provides permissions select Attach existing policies directly and select the Policy that enables to work only with specific S3 bucket, if you want this user to be just a Worker, like an external programmer. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/3_users/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/3_users/ Clumsy way (optional) Google sign in setup Adding users Admin users Removing users This chapter covers user management and Google OpenID sign-in setup. Clumsy way (optional) Here’s a command to add a user to Ubuntu server: sudo useradd -m -d /pfs/home/<user> <user> sudo passwd <user> # will ask to enter a password Once you’ve added a user it should be possible to login into Workbench using this user’s Ubuntu credentials. https://handbook.gradientmetrics.com/handbook/handbook-style-guide/using-markdown/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/handbook/handbook-style-guide/using-markdown/ Introduction Headings Paragraphs, Breaks, and Horizontal Lines Wrapping Text Horizontal Lines Emphasis: Bold and Italic Links Inline Links Relative Links Mailto Links Lists Ordered lists Unordered Lists Images Videos Introduction Markdown is a way to style text on the web. You control the display of the document; formatting words as bold or italic, adding images, and creating lists are just a few of the things we can do with Markdown. https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/issue-types/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/issue-types/ Issue Types Issue types in Jira are used to categorize and differentiate issues based on their purpose or nature, or to meet your team’s specific needs. Before you start editing your workflow, you need to determine if you need new issue types in the Jira system to create and launch your workflow. For example, when creating the existing onboarding workflow, each issue type has to have its own workflow attached to it. https://handbook.gradientmetrics.com/operations/finance/vendor-invoices/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/vendor-invoices/ Introduction Invoice Submission Process Introduction To streamline the invoice submission process, all invoices will be processed through the below steps and submitted to Gradient’s AP Platform for processing and payment. All invoices must be submitted through this process for proper tracking and processing. Invoice Submission Process Gradient employee receives invoice from Vendor by email. This is commonly the Project Manager for a client project, or a department lead (such as Operations). https://handbook.gradientmetrics.com/products/market-sizing/analyzing-tam-sam-data/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/market-sizing/analyzing-tam-sam-data/ TAM Step 1: Define the total addressable market (TAM) for your product or service Step 2: Calculate the TAM Via Quota Groups Step 2a: Determine the size of each demographic segment in the TAM Step 2b: Determine the revenue potential for each customer segment Step 2c: Sum the revenue potential for each customer segment SAM Step 1: Define the serviceable available market (SAM) for your product or service Calculating Market Penetration and Average Expenditure via WTP Step 2: Calculate the SAM Via Quota Groups Step 2a: Determine the size of each demographic segment in the SAM Step 2b: Determine the revenue potential for each customer segment Step 2c: Sum the revenue potential for each customer segment TAM Step 1: Define the total addressable market (TAM) for your product or service This is the total market demand for the client’s product or service, regardless of any existing competition or market share. https://handbook.gradientmetrics.com/sales/sales-playbook/channel-partners/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-playbook/channel-partners/ Orienting on Agency Customer Base: Orienting on Agency Vendor Selection Process: Orienting on Agency Customer Base: Can you give me a better sense of the types of clients you do research for and what that research generally entails? What is the cadence of the research and how much do you support internally vs. work with external partners? Who on the team is generally responsible for providing the research? https://handbook.gradientmetrics.com/operations/finance/contractor-invoices/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/contractor-invoices/ Introduction Remote.com Management Adding New Contractor Process Justworks Management Adding New Contractor Process Introduction This document serves as the process for payment of contractor invoices. Gradient uses two different platforms for paying contractors: Remote.com and Justworks. Generally, Remote is used for payment of Gradient “full-time” contractors and non-US based contractors hired for smaller projects, whereas Justworks is used for payment of US-based contractors hired for smaller projects (such as the survey reviewer panel). https://handbook.gradientmetrics.com/gradient/company-guidelines/correspondence-hygiene/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/correspondence-hygiene/ Networks Are Important Networks Are Non-Linear Principles Always Follow Up Always Be Helpful Always Care Don’t Trust Yourself to Remember Understand Their Problem Better Than They Do Listen, but Guide Always Have a Plan Don’t Let the Energy Flag Always Be On-Point Summarize the Meeting You Just Had Always Take Notes Always Guide to Next Steps Networking Events Quantity > Quality Networks Are Important There is no substitute for building a strong personal network. https://handbook.gradientmetrics.com/operations/data-security/pii-policies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/data-security/pii-policies/ Overview Personally Identifiable Information Our Data Data Storage and Transmission Access Policies Who Has Access to Personally Identifiable Information? Overview Where absolutely needed, Gradient will collect personally identifiable information (PII) from survey respondents to support analytical needs. This comes with a great deal of responsibility to keep these data safe, and this part of our IT security and Data Handling policy sets out the roles and responsibilities when dealing with sensitive data. https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-templates/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-templates/ https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/github-builds/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/github-builds/ Overview Workflow yaml files Triggers Initial job setup Environment variables Steps Overview Before pushing package updates to github you should always run R CMD check to make sure that you did not break any CRAN standards and that all the tests continue to run well. But still, to be on the safe side, we run R CMD checks in an automated way using github actions. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/4_global_settings/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/4_global_settings/ tmp folder Rprofile.site This chapter covers global system settings that will be applied to all users. tmp folder By default Ubuntu uses “/tmp” folder for temporary storage. In our current setup we use rather small internal ec-2 storage. So there’s a certain risk that this storage will be filled to capacity, especially if somebody happens to be doing Stan modeling. Therefore it makes sense to set the default temporary storage to a folder on the mounted persistent drive. https://handbook.gradientmetrics.com/sales/sales-playbook/demo/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-playbook/demo/ The Basics | Pre Demo Prep The Demo Itself | Things To Focus On The Demo Itself | Things To Avoid The Basics | Pre Demo Prep Be prepared: Research the company, attendees, and notes from prior conversations to tailor the demo to the company/individual’s specific needs. This is too often overlooked - respect your prospect’s time and keep it to the point. https://handbook.gradientmetrics.com/newsletter/analysis-visualizaiton/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/analysis-visualizaiton/ https://handbook.gradientmetrics.com/product-team/devops/aws/our-machines/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/our-machines/ Regular Machines Listing active AWS instances Qualtrics backups Dynata collection progress Docker-based Machines APIs Overquotas API There are a couple of EC2 machines that are running on a daily basis that provide various monitoring purposes. This page lists the machines, their credentials and their purpose. They mostly make use of Slack Notifications to inform team members of updates. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/outputs-and-statistics/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/outputs-and-statistics/ Design diagnostics Statistics Standard errors Frequencies and Pairwise Frequencies Overlaps Extracting Design Nice orange messages Design diagnostics So we managed to fill all the inputs, click the calculate button, and we need to interpret the output somehow. Output Preview Statistics In the first tab, you will see a bunch of statistics. The most important for us are: D-error (we want to minimize that) and balance (there are 4 different balance fields) – we want to keep it close to 1, but we assume that anything above 0. https://handbook.gradientmetrics.com/product-team/coding/positron_setup/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/positron_setup/ This page is not meant to be an exhaustive guide to setting up Positron, but rather a collection of setup hints that are easy to miss if you do not already know them. Quarto extension At the time of writing, the most recent Quarto extension version, 1.131.0, introduces a noticeable delay when executing code from an .Rmd document with Cmd/Ctrl + Enter while the cursor is inside a code chunk: it can take around a second before the code is actually sent to the console. https://handbook.gradientmetrics.com/products/power-calculations/analysis-power/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/power-calculations/analysis-power/ Introduction & the goal Motivating Example A more complicated example Introduction & the goal Statistical power is the probability that, If there is an effect That your model will detect that effect At a given confidence level Typically, power will depend on: The size of the effect (the larger the effect, the easier it is to gain statistical power) The sample size (the larger the sample size, the easier it is to gain statistical power) The amount of “noise” (the more noise, the harder it is to gain statistical power) Motivating Example Let’s take an easy example. https://handbook.gradientmetrics.com/gradient/onboarding/projects/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/onboarding/projects/ Project Lifecycle New Business Development Upfront Synthesis Project Management Sprint Planning Deliverables Kaizen Project Lifecycle New Business Development Currently, the majority of our business comes from referrals or repeat business. Although all Gradient team members are empowered and encouraged to seek out new business, the Head of Research and Research Managers have a primary role in securing opportunities. https://handbook.gradientmetrics.com/projects-clients/project-startup-workflow/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/project-startup-workflow/ Create a Codename Create a Project in JIRA 1. Create a New Project 2. Check Screen Settings for the New Project 3. Customize the Project Description, Avatar, Etc. 4. Create a JIRA Board Integrate into Slack Create a Google Drive Folder Set Up AI Project Folder with Materials Create a Shared Folder Upload Materials to the Folder Prepare Project Playbook Add Bookmarks to Slack Create a Google Group Create & Add a Roadmap to JIRA Consider Releases Consider Reviews Add Recurring Issues (If Applicable) Post Mortem Setup Pre Mortem Think About the Gradient Machine Add to Quickbooks A Step-by-Step Guide to Setting Up New Projects https://handbook.gradientmetrics.com/operations/people-policies/vacation/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/vacation/ Why We Have a PTO Policy Policy Conditions Request Process Actions To Take Upon PTO Approval Unused PTO Sick Leave Is Not PTO Why We Have a PTO Policy Gradient values work-life balance and has instituted this PTO policy in an effort to ensure our team is rested, taking healthy time off from work, and that the rest of the team’s bandwidth is manageable while people are out. https://handbook.gradientmetrics.com/research-team/survey/survey-weighting/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey-weighting/ Motivation What are survey weights? Weighting methods Trade-offs Example Inverse probability weighting Using models other than glm Oversample weights Example Motivation Oftentimes in market research, we try to make inferences about a population. What proportion of American voters agree with a statement? But of course, we can’t conduct a census. For most populations, that’s much too expensive, infeasible, and just downright silly. https://handbook.gradientmetrics.com/products/misc-analyses/survival-analysis/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/misc-analyses/survival-analysis/ https://handbook.gradientmetrics.com/products/maxdiff-experiments/synthesizing-maxdiff/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/maxdiff-experiments/synthesizing-maxdiff/ Summary Metrics Output Table Output Plot Summary Metrics After the analysis determines the utility coefficients, outputs and deliverables can be prepared to showcase the findings of the study. The utilities are the building blocks of all of the summary metrics. The core summary metrics that typically accompany MaxDiff analysis are detailed below: Preference Share: The preference share is the measurement of the probability that an item would be chosen over another if a respondent was asked to select the best from all options. https://handbook.gradientmetrics.com/products/segmentations/synthesizing-segmentations/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/segmentations/synthesizing-segmentations/ https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/using-mockups/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/using-mockups/ Why use mockups? Data as an argument Data needs to be downloaded or processed within function Why use mockups? In some cases we might be unable to test functionalities. This might be due to long time of computation, or the function needs inputs that are stored on an external server. This also includes functions that deal with data that change in time (for instance data provided by an external API). https://handbook.gradientmetrics.com/handbook/handbook-style-guide/using-shortcodes/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/handbook/handbook-style-guide/using-shortcodes/ Shortcodes Sometimes Markdown falls a bit short, and this is where Hugo’s Shortcodes come in. Shortcodes make our documents a lot cleaner when we need to use raw HTML code. For example, markdown has no support for embedding videos through an <iframe> tag, but we can create a shortcode to circumvent this limitation. A shortcode can be called by calling {{< shortcodename parameters >}}. Shortcode parameters are space delimited, and parameters with internal spaces can be quoted. https://handbook.gradientmetrics.com/product-team/coding/coding-standards/naming-conventions/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/naming-conventions/ General Data Frames Constants Examples Other variables Examples Functions Examples This guide explains how we name our code variables at Gradient. This refers to every R script we will ever create. Read through this and adhere to our protocol to improve collaboration! If a script in a Pull Request does not adhere to these naming rules, link to this page and let the coder update the names. https://handbook.gradientmetrics.com/product-team/coding/package-development/naming-rules/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/naming-rules/ General Data Frames Constants Examples Other variables Examples Functions Examples This guide explains how we name our code variables at Gradient. This refers to every R script we will ever create. Read through this and adhere to our protocol to improve collaboration! If a script in a Pull Request does not adhere to these naming rules, link to this page and let the coder update the names. https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/fields/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/fields/ Custom Fields Custom fields in Jira are used to capture additional information about issues beyond the default fields provided by Jira. To create a new custom fields: Click on the Jira settings gear icon in the top right corner of the screen. Select Custom Fields from the sidebar menu. Confirm the custom field you are considering creating is not already on the list. Click on the Create custom fields button in the top right corner of the screen. https://handbook.gradientmetrics.com/operations/finance/accounts-payable/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/accounts-payable/ Introduction Process Introduction This document serves as the process for all external invoices submitted to Gradient for payment. Process Invoice receiver submits invoice per the Invoice Submission process (Handbook) and Vendor Invoice Submission form. Airtable notifies billing@gradientmetrics.com (and any other designated email address) of the submission to the Finance base > Vendor Invoices table. You are able to click into the record directly from the email. https://handbook.gradientmetrics.com/gradient/company-guidelines/calendar-best-practices/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/calendar-best-practices/ Introduction Personal Calendar Out of Office Availability Working Hours Meeting Notifications Meetings with Teammates Recurring Meetings Visibility Slack Integration Gradient Schedule Calendar Resources Introduction As a fully remote company with team members all over the world in multiple time zones, it is important that our Google Calendar hygiene is close to impeccable here at Gradient. This document sets out best practices and general guidelines for Google Calendar use. https://handbook.gradientmetrics.com/research-team/survey/penalty-score-data-quality/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/penalty-score-data-quality/ Purpose Core Principles Ownership & Responsibilities When Penalty Scores are Specified Where Penalty Scores are Documented Required Checklist Items (RM-Owned) Standard Penalty Score Components 1. Speeding Default Rule 2. Module-Specific Speeding (Optional) 3. Straightlining Default Rule 4. Low-Quality Open-Ends Default Rule 5. Inconsistent or Implausible Responses (Optional) Suggested calibration Removal Thresholds Default Rule Important Notes Purpose This page documents Gradient’s standard approach to penalty scores: how we define, communicate, and apply penalties to identify and remove low-quality survey responses. https://handbook.gradientmetrics.com/projects-clients/project-closeout-workflow/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/project-closeout-workflow/ Research Manager Closeout Confirm Jira contractor issues are closed out Client Debrief Archive Deliverables in Repository and Drive Clean-up Data Deletion Testimonial/Referral Request Operations Closeout Contractor Closeout Confirm Data-Related Actual Costs Invoice the Client Infrastructure Closeout Jira: Remove Recurring Issues Release All Releases Jira: Move to Closed Projects Archive Slack channel Post Mortem Code Closeout Clean Up Project Directory Document The Project Close Project on Github A step-by-step guide for closing out completed projects. https://handbook.gradientmetrics.com/projects-clients/contractor-effort-tracking/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/contractor-effort-tracking/ Introduction Contractor Profile Manual Issue Creation Introduction Gradient uses metrics to determine the profitability of client projects, manage bandwidth, and evaluate effort by Gradient team members. This contractor effort tracking process was created to make sure we are including contractor effort in our performance metrics. Contractor Profile Gradient has created a singular Contractor profile in Jira to be used to assign contractor issues and track effort in Gradient metrics. https://handbook.gradientmetrics.com/products/conjoint-experiments/conjoint-simulator/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/conjoint-simulator/ Introduction How does a Market Simulator work? Creating a Market Simulator at Gradient Getting Started Uploading your data Changing the Layout and other Aesthetics Updating Control Boxes Updating Market Share Calculation Adding Filters Adding Weights Using Custom R Code Adding Conditionality Publishing your Dashboard Adding Users & Access Restrictions Introduction The market simulator is usually considered the most important tool resulting from a conjoint analysis project. https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-workflows/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/projects-clients/deliverables/deliverable-workflows/ https://handbook.gradientmetrics.com/product-team/coding/coding-standards/file-saving-conventions/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/file-saving-conventions/ Project files in a Repository R Data R Objects Project files in a Repository Our projects all follow the same structure as explained in Coding at Gradient Never use any of the following Unicode characters in your file names: < (less than) > (greater than) : (colon) " (double quote) / (forward slash) \ (backslash) | (vertical bar or pipe) ? https://handbook.gradientmetrics.com/product-team/coding/package-development/file-saving-rules/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/file-saving-rules/ Project files in a Repository R Data R Objects Project files in a Repository Our projects all follow the same structure as explained in Coding at Gradient Never use any of the following Unicode characters in your file names: < (less than) > (greater than) : (colon) " (double quote) / (forward slash) \ (backslash) | (vertical bar or pipe) ? https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/github-builds-fixing/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/github-builds-fixing/ General recommendations Difficulties in preconfiguring the build environment Inconsistencies between your working environment and the one on GitHub Temporary issues Resolving Build Issues: An Example General recommendations To begin with, it is crucial to first ensure that the build functions properly in your normal working environment, that is in RStudio. If the build runs well in your working environment yet encounters problems on GitHub, the underlying cause is almost always related to one of the following: https://handbook.gradientmetrics.com/gradient/company-guidelines/meetings/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/meetings/ Start on Time, End on Time Shoot the Shit, But Not for Too Long Have an Agenda, Have a Goal Formal or Informal Set the Context Know the Goal Going In Someone Should (Almost Always) Be Sharing Their Screen Take Notes, Preferably in Public Use Loom to Record the Meeting Be High Energy, Don’t Leave Dead Space Call on Specific People Versus Asking the Room It’s Okay to Leave Time for the Reading Keep an Eye on the Clock Recap the Next Steps and Who’s Doing What Start on Time, End on Time You should assume that everyone is back-to-back. https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-crunch/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/rm-best-practices/intro-crunch/ What is Crunch.io? Navigating Crunch.io Exploring summary data Data weights Numerical data Adding filters Temporary filters Saved filters Exploring crosstabs Advanced cross tabs - Multitables Creating plots Single variable plots Crosstab plots Saving plots and tables Exporting and sharing decks Exporting decks Sharing decks Manipulating data Recoding variables Combining categories Creating categorical variables Managing users What is Crunch. https://handbook.gradientmetrics.com/operations/employment-law/lawful-off-duty-activities/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/employment-law/lawful-off-duty-activities/ The Company will not discriminate against any employee who engages in lawful off-duty activities (including the use of legal consumable products, legal political activities, legal recreational activities, or membership in a union or exercise of a right granted under the law), outside of the workplace, during non-work hours, and without the use of the Company's equipment or other property. https://handbook.gradientmetrics.com/operations/employment-law/otherleave/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/employment-law/otherleave/ The Company will not discriminate against a victim of domestic violence who requests to use paid time off in order to attend to issues pertaining to the domestic violence, such as court appearances, doctor’s visits, and seeking other assistance. If you wish to take such leave, please contact Kyle Block or Stefan Musch. All requests for such leave will be treated confidentially on a need-to-know basis and no employee will be retaliated against for requesting or taking leave under these circumstances. https://handbook.gradientmetrics.com/operations/finance/medical-insurance/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/medical-insurance/ Introduction TakeCommand Management Process Introduction This document serves as the process for reimbursing full-time United States based Gradient staff for their medical benefits using TakeCommand and Justworks. TakeCommand Management Due to the nature of Gradient’s small size, we use an ICHRA platform, TakeCommand, to manage our medical benefits. Employees can either select their insurance through TakeCommand’s “market” or “bring your own.” Remember to add new employees (if they are US-based) to the system ASAP because it can take some time to get them set up. https://handbook.gradientmetrics.com/research-team/survey/survey-mrp/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey-mrp/ Introduction & the goal How does MRP work? Example of MRP Applications Multi-level Regression Accuracy Poststratification Step 1 Step 2 Extending the simulation Summary Source: vignettes/mrp.Rmd Content based on: Introduction & the goal Multi-Level Regression and Poststratification (MRP) is a technique that is used to adjust model estimates for known differences between your sample and the target population. https://handbook.gradientmetrics.com/operations/people-policies/leave-policies/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/leave-policies/ The following policies reflect Gradient’s non-PTO time off. Please find the PTO policy here. Bereavement Leave Parental Leave Birthing Parent Leave Non-Birthing Parent Leave Sick Leave and Dependent Care Short Term Disability Leave Bereavement Leave When an employee experiences the death of a loved one, they may take bereavement leave. Bereavement leave provides time to grieve or make arrangements after the death of a personal relationship such as a family member, chosen family, close friend, miscarriage, or family pet. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/5_r_libraries/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/5_r_libraries/ R library folder renv folders Initial package installation This chapter is dedicated to our R library setup. We want to store R libraries along with renv cache on the mounted persistent drive. In the future it may be useful to keep multiple versions of R installed. R library folder We will use R installations on ec-2 internal storage with global shared libraries on the mounted drive. https://handbook.gradientmetrics.com/products/market-sizing/market-sizing-reference/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/market-sizing/market-sizing-reference/ The best overall guide comes from Michael W Silverman. Differences between TAM, SAM, and SOM From the Power MBA From Hubspot Pros and Cons of Different Market Sizing Approaches From Cognism From Antler From Towards Data Science From Kadence Example Calculations From Toptal From Market Research Methods From beeso.io From Hubspot From Pear https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/sample-size/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/sample-size/ What sample size do I need? Sample size when using prohibitions What sample size do I need? Well the basic rule of thumb is 200 per each subgroup or at least 300 total, whichever is greater. Another rule of thumb is to use equation: \[n*t \geq 10 * \frac{k}{p}\] Where n is the sample size, t is the number of choice sets, k is the total number of levels minus the total number of attributes, and p is the probability of the least frequently chosen alternative. https://handbook.gradientmetrics.com/sales/sales-process/stage-proposal/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-process/stage-proposal/ Pre-call (How we prep for this stage) What are we trying to learn? What does great look like (what constitutes success)? Definition The proposal stage is entered into once you have received “Product Yes” or “Vendor of Choice” designation by the key decision makers. At this stage the prospect is fully bought in on working with Gradient Metrics, has a full understanding of the project scope, methodology, support structure, deliverables, timelines, and intended impact. https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/screens/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/tools/JIRA/jira-workflows/screens/ Screens Screens in Jira are used to display fields and information to users when they interact with issues. How to create screens: Click on the Jira settings gear icon in the top left corner of the screen. Click on Issues Select Screens from the sidebar menu. Click on the Create screen button in the top right corner of the screen. Give your new screen a name and description, then click Add. https://handbook.gradientmetrics.com/newsletter/writing-stories/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/writing-stories/ Introduction Timeline Picking Stories to Run Plot Specs Writing a Story Writing a Product Story Front Matter Subject Line Preheader/Preview Text Email Introduction Whiteboarding Introduction Writing a newsletter story requires you to be creative, precise, analytical, and at least a little bit funny. It is not easy, and you should plan on a story taking a couple of hours to write. https://handbook.gradientmetrics.com/research-team/survey/cloud-research/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/cloud-research/ Introduction Getting started with CloudResearch Step 1. Basic Information Step 2. Describe HIT Step 3. Setup & Payment Step 4. Demographics Step 5. Data Quality Step 6. Worker Approval Step 7. Worker Requirements Launching your survey Step 8. Monitor Launch Introduction In the past, testing our Qualtrics surveys was easy—we would simply press the Generate Test Responses button and have a bunch of test cases to work with. https://handbook.gradientmetrics.com/research-team/survey/survey-crunch/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/research-team/survey/survey-crunch/ Introduction to Crunch - An Interactive Survey Dashboard Platform Crunch.io tutorials A typical Crunch workflow at Gradient Example dataset Logging in Managing Datasets Locking and unlocking Hiding variables Merging variables into an array Changing names and descriptions Arranging variables and directories Forking and Merging Interaction through web interface Crunch Tips & Tricks Non-standard Crunch Functions Opening a crunch dashboard from an R session Sorting variables inside a folder Managing rows Managing columns Forking and merging Source: vignettes/crunch. https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/additional-checks/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/products/conjoint-experiments/Conjoint-Designs/creating-designs-in-displayR/additional-checks/ Additional things to do Check if you have not missed anything Do the conjoint tasks look reasonable? Additional things to do After you check the numbers and everything seems perfect in terms of the numbers, there are two more checks you should always do. Check if you have not missed anything The simplest way to do it, is to set up filters in your excel sheet with design. https://handbook.gradientmetrics.com/operations/employment-law/ada/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/employment-law/ada/ The Americans with Disabilities Act (ADA) is a comprehensive federal civil rights law in the U.S. that specifically protects individuals with physical and mental disabilities. State and local law provide comparable protections. Individuals are protected under the ADA if any of the following conditions exist: They currently have a physical or mental condition that substantially limits their ability to normally conduct a major life function (walking, seeing, hearing, breathing, etc. https://handbook.gradientmetrics.com/product-team/devops/aws/ssh-keys/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/ssh-keys/ For security considerations, we strive to prevent IAM users from having both console access and security keys enabled. In practice, this means that each individual requiring console access via the web interface will have a corresponding IAM user without security keys enabled. The Workbench server should be associated with a role that provides S3 access without explicitly storing keys on the instance. For more details, refer to this section. However, if access keys are indispensable for a particular task and no alternatives are available, one should establish a dedicated IAM user for that specific task, ensuring that permissions are judiciously set for this user. https://handbook.gradientmetrics.com/operations/finance/ramp-instructions/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/ramp-instructions/ Introduction Communication Preferences Credit Cards Using Your Card Recording Expenses Virtual Card Requirements Receipt Memo QuickBooks Category QuickBooks Customer/Job Billable Physical Card Requirements Adding Receipts Accidental Charges Unrecognized Charge Alert Thresholds Reimbursements Reimbursement Requirements Ramp Best Practices Ramp Tips & Tricks Introduction Gradient uses the Ramp platform to manage employee-incurred expenses. During the onboarding process, you will receive an email invitation to accept your new virtual credit card and activate your Ramp account. https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/internal-cran/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/internal-cran/ Overview Our Internal CRAN is a repository of all our internal packages, similar to the original CRAN, but exclusive to us and not open to the world. It allows us to easily install our packages using the standard install.packages() command and contains all versions of our packages. While it only contains sources and not binaries, resulting in longer installation times, it is easier to maintain this way. The latest versions of our packages are automatically added to our internal CRAN through a Github build process. https://handbook.gradientmetrics.com/newsletter/launching/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/newsletter/launching/ Timeline Sending the Newsletter with Beehiiv Log In Create New Post Import Content Post, Email, and Website Settings Sending the Test Email Sending the Newsletter Canva Workflow Log in Canvas Size Insert the Plot Create the Annotations Insert the Plots into the Newsletter Timeline We run two newsletters per month. The first usually launches during the first week of the month, and the second launches two weeks later. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/6_package_installation_issues/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/6_package_installation_issues/ Problems with *.so libraries Cannot open the connection cmdstanr Other individual cases Problems with *.so libraries If you see an error like this: Error: package or namespace load failed for ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/pfs/R/libs/4.2.0/sf/libs/sf.so': libproj.so.15: cannot open shared object file: No such file or directory Most likely it means that a package was installed from a binary. https://handbook.gradientmetrics.com/operations/people-policies/remote-work-stipend/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/remote-work-stipend/ Policy Purpose Eligibility Currency Treatment How Purchases Are Counted Tracking, Documentation, & Payment Methods Remote Work Budgets New Employee Startup 1. Computer Hardware 2. Computer Warranty 3. Office Equipment and Furniture Regular Computer Replacement Annual Workspace Stipend Budget Replenishment Timeline Exceptions for Urgent Needs Policy Review Policy Purpose Gradient is a fully remote company. https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-best-practices/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/code-testing/testing-best-practices/ Testing is crucial for developing robust and safe code. Nonetheless, writing automated test is difficult and a time consuming process. Here is short summary of things you may want to consider before you start writing tests. Keep functions as small as possible. Functions should deal with one single thing, and they should deal with it good. For instance consider separating data wrangling from plotting or modeling. Make some sanity checks within function (e. https://handbook.gradientmetrics.com/gradient/company-guidelines/video-chat/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/gradient/company-guidelines/video-chat/ Some guidelines for being impressive while on calls: If you’re typing, please stay on mute. Typing comes out pretty loud and adds a level of background noise that you can’t hear. Please don’t eat when you’re on a call, unless we’re super close with the client. Please don’t eat when there is new business on the line, or we are trying to impress people that will make decisions about whether or not we will be given new business. https://handbook.gradientmetrics.com/product-team/coding/package-development/multicomponent-functions/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/multicomponent-functions/ Decomposition Checking inputs Decomposition If your function executes multiple non-trivial steps and these steps look like they could be encapsulated into separate functions on their own - then they almost always should. This is a bit vague instruction but it’s hard to put it more precisely without formalizing too many things thus making it practically unusable. Basically we have to keep in mind the ultimate goal here - we want to have functions that are easy to maintain. https://handbook.gradientmetrics.com/operations/finance/ramp-admin/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/finance/ramp-admin/ Introduction General Management Cards Changing Limits Locking Cards Default Accounts Account Mapping Reimbursements Introduction This document serves as documentation of administrative management of the Ramp platform. General Management Cards Changing Limits Go to Cards Click on card to change At the bottom of the new screen, click Edit Update Amount Save Changes Locking Cards Go to Cards Select card to lock At the bottom of the new screen, click Lock Card will update with locked information Can be unlocked the same way History of locking etc. https://handbook.gradientmetrics.com/marketing/media-outreach/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/marketing/media-outreach/ The goal of this Media Outreach Plan is to develop a small network of press contacts that we can take story ideas to directly. The goal of that plan is to be able to get more articles published without having to go through external data sources. The goal of getting more articles published is to develop our brand The goal of developing our brand is to Close more existing conversations Open more conversations Develop a distribution network to offer novel products Media Tracker Below pages are both public. https://handbook.gradientmetrics.com/product-team/coding/coding-standards/our-standard-libraries/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/coding-standards/our-standard-libraries/ Preferred / go-to packages Other noteworthy packages As a team of coders — it’s very healthy and good practice to use the same set of libraries. This improves our ability to collaborate and the readability of our code. This document showcases the libraries that we picked out as a team, and should always be used over their alternatives if the same results can be achieved. https://handbook.gradientmetrics.com/sales/sales-process/stage-contracting/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/sales/sales-process/stage-contracting/ Pre-call (How we prep for this stage) What are we trying to learn? What are we trying to show? What does great look like (what constitutes success)? Contracting Playbook Pre-Contracting Phase: Contract Initiation: Redlines Management: Redlines Negotiation: Negotiation Specifics: Definition The Contracting Stage is the phase in the sales pipeline where negotiations are nearing finalization (or finalized), and a formal agreement or contract is drafted, reviewed, and sent for signature. https://handbook.gradientmetrics.com/operations/people-policies/termination/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/operations/people-policies/termination/ Voluntary Termination The Company will consider that you have voluntarily terminated your employment if you do any of the following: Elect to resign from the Company; Fail to return from an approved leave of absence on the date specified by the Company; or Fail to report for work without notice to the Company for three consecutive days. Involuntary Termination You may be terminated involuntarily for reasons that may include poor performance, misconduct, or other violations of the Company’s rules of conduct as set forth in the Employment Law section of the handbook. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/7_testing_server/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/7_testing_server/ System state Turn off Open ID Known issues To properly try potential Workbench server updates you have to create a testing server. It’s not a good idea to test on our main server! Although for the tests to be as relevant as possible it’s better to have your testing environment closely resemble the production one. In our case in the first place this implies the state system drive of the server instance. https://handbook.gradientmetrics.com/product-team/coding/package-development/data-pronoun/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/data-pronoun/ Reasons to use .data pronoun tidyselect > 1.2.0 and selecting Reasons to use .data pronoun We are extensively using tidyverse framework in our functions. You can find calls to dplyr’s or tidyr’s common functions in the majority of our code. Many of the functions use data-massking (see rlang data masking). In short, it means that you use the unqouted names passed to the function’s parameters. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/8_job_launcher/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/8_job_launcher/ Introduction Setup Overview Considered but Unchosen Setup Options Slurm instead of Kubernetes Fargate for EKS Other storage options Prerequisites Tutorials Cloud9 Setup Creating EKS Cluster with Primary Node Group Karpenter RStudio service account FSx PVC RStudio configuration rserver.conf launcher-mounts launcher.conf launcher.kubernetes.profiles.conf launcher.kubernetes.conf kubernetes.launcher.token RStudio image Troubleshooting Launcher logs LD libraries Introduction The RStudio Server typically meets our needs, but occasionally running complex models can overload it, affecting user experience. https://handbook.gradientmetrics.com/product-team/coding/package-development/import-tags/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/coding/package-development/import-tags/ Using roxygen tags to import packages and functions Where should we put the import tags Using roxygen tags to import packages and functions We all stand on the shoulders of giants AKA we do extensively use functions written by other developers to create our own code. Using functions from other packages in our own packages, can be made in 3 ways: You can directly refer to package and function using package::fun() syntax in the code. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/9_monorepo_mirror/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/9_monorepo_mirror/ We are accustomed to having a separate copy of our monorepo for each project we work on. However, since our repository is quite large, doing git clone every time can be incredibly slow. To address this issue, we’ve set up an on-server monorepo mirror. This is a centralized copy that is regularly synchronized with GitHub. This allows us to quickly create new copies from the mirror instead of cloning the entire repository each time. https://handbook.gradientmetrics.com/announcements/prior-announcements/2025-announcements/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/announcements/prior-announcements/2025-announcements/ 5 December: Introducing Gradient's Remote Work Stipend Policy ↕ Hi everyone! We’re excited to introduce Gradient’s new Remote Work Stipend Policy, beginning 1 January, 2026! 💻 The Remote Work Stipend Policy includes three types of budgets designed to support of a successful remote work setup. These budgets cover expenses around: Setting up your home office upon hire, Replacing your computer every five years, and An annual stipend for ongoing improvements to your workspace, such as buying new headphones or using a coworking space. https://handbook.gradientmetrics.com/announcements/prior-announcements/2024-announcements/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/announcements/prior-announcements/2024-announcements/ 19 November: Release Automation in Jira ↕ Thanks to the visionaries Stefan, Brendon, and Cory, we have implemented an automation in Jira that will populate standard project releases when a new project is created! This removes the need for you to manually create releases for each client project, and allows us to see metrics more clearly across the same types of releases. Visit Jira Hygiene for a list of the releases and example lists of the types of issues that can fall within them. https://handbook.gradientmetrics.com/product-team/devops/aws/sagemaker/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/aws/sagemaker/ This page is based on this notion page and multiple videos. Videos Real time endpoints Batch transformations Overview Pricing Quick Usage What is happening under the hood? Going top-bottom Going bottom-top Notes Videos Real time endpoints Batch transformations Overview AWS SageMaker is a platform on Amazon Web Services meant to create Machine Learning models and serve them as a paid service over an API. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/10_submitting_diagnostics_reports/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/10_submitting_diagnostics_reports/ When submitting a support ticket to Posit Support regarding an issue with Workbench, it is advisable to include a diagnostics report. Typically, the support team will request this report if it is not initially provided. Below, we outline an efficient method for submitting this report. Access your server instance via SSH. Execute sudo rstudio-server run-diagnostics. You’ll see a message indicating the diagnostics report has been generated, along with its file path, e. https://handbook.gradientmetrics.com/posts/initial-release/ Tue, 28 Jan 2020 00:00:00 +0000 https://handbook.gradientmetrics.com/posts/initial-release/ We’re proud to present the first version of the Gradient Handbook! The Gradient team handbook is the central repository for how we run the company. As you know, we’re 100% remote—which means that we have a ton of written content. The handbook was our way of ensuring that all of our company information was accessible to everyone regardless of when they became part of the team. It’s our Single Source Of Truth (SSOT) https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/codebase-maintenance-overview/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/codebase-maintenance/codebase-maintenance-overview/ Almost all coding we do ends up in our big main github repository, or as we call it, the monorepo. There are a few exceptions, but these are not going to be considered in this section of the handbook. Because the monorepo is so huge and because multiple people actively work on it in parallel, it is critically important to keep it in order. To achieve this we have a set of rules and processes we follow. https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/0_overview/ Mon, 01 Jan 0001 00:00:00 +0000 https://handbook.gradientmetrics.com/product-team/devops/Rstudio_Workbench/0_overview/ Here are RStudio guides for Workbench: a small one, and a big one. The problem is that unless you are an experienced devops engineer you may find some (a lot of) instructions unclear. Some of them may even be misleading. This set of pages is written to serve as a supplement to RStudio guides. It covers details relevant specifically to our way of setting up Workbench and simply things that might be unclear in general.