{"id":278,"date":"2021-08-31T14:40:23","date_gmt":"2021-08-31T14:40:23","guid":{"rendered":"https:\/\/fde.cat\/?p=278"},"modified":"2021-08-31T14:40:23","modified_gmt":"2021-08-31T14:40:23","slug":"a-peek-at-datoramas-virtual-sandbox","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2021\/08\/31\/a-peek-at-datoramas-virtual-sandbox\/","title":{"rendered":"A Peek at Datorama\u2019s Virtual Sandbox"},"content":{"rendered":"
\"\"<\/figure>\n

What is a\u00a0Sandbox?<\/h4>\n

A Sandbox is an isolated environment where you can safely experiment and test changes that you intend to apply to your live environment. The idea is to learn about the implications of these intended changes in advance, in order to decide on how to best conduct them, without actually causing any harm to the live environment in the\u00a0process.<\/p>\n

How is This Usually Addressed?<\/h4>\n

The way in which most companies solve the sandbox requirement is by copying the live environment onto a completely separate environment. Testing and experimenting then takes place on the copied environment, with no impact on the live one. This approach has a few drawbacks:<\/p>\n

    \n
  1. Copying the data from one environment to another, along with the entire configuration, is a very time consuming process.<\/li>\n
  2. While working on your sandbox environment, your live environment gets constantly updated with new data and configuration changes. This means that within time, the sandbox environment will no longer truly represent the production environment.<\/li>\n
  3. In most cases, the data is only partially copied from the live environment to the sandbox. This data deficiency in the sandbox makes it impossible to truly appreciate the full implications of your sandbox\u00a0actions.<\/li>\n<\/ol>\n

    How is Sandbox for Datorama Different?<\/h4>\n

    Rather than having two separate environments, our solution is based on having two separate modes<\/em>, a sandbox mode and a production mode, within the same environment, where changes made in the sandbox mode have zero impact on the production mode. Having the two modes in the same environment eliminates the need to copy data from one environment to another, meaning the data is never partial and the setup process is always instantaneous. Furthermore, the two modes never differ with the passage of time. They are both perfectly aligned, 100% of the time. The way in which we were able to achieve this is by making the distinction between sandbox and production right down to the entity level<\/strong>, rather than the environment level. So, any given list of entities, be it dashboards, reports, pages, data streams, etc., is identified as either being production or sandbox entities:<\/p>\n

    \"\"<\/figure>\n

    From there, we just make sure that the right entities are accessible from the relevant environment. So, within production, only the production entities are accessible, since all of the Sandbox entities are filtered out. This means that anything you do within the sandbox has no impact on production whatsoever. Within the sandbox environment, you have both production and sandbox entities. This is because, in addition to adding new sandbox entities, you may also want to edit existing production ones to see how they interact:<\/p>\n

    \"\"<\/figure>\n

    The Data Stream Entity\u200a\u2014\u200aLoading & Querying sandbox\u00a0data<\/h4>\n

    In order to explain Data Querying in sandbox mode, we will examine one of the main entities a customer has on their workspace\u200a\u2014\u200aData\u00a0Stream.<\/p>\n

    A data stream is the configuration entity that is responsible for the data ETL process. It is responsible for pushing\/pulling data from the different data providers, transforming the data according to the user definition, and loading it to the Datorama data warehouse.<\/p>\n

    To allow differentiating between the different streams data, the Data Stream ID is part of every row of\u00a0data.<\/p>\n

    When the user edits a Data Stream in sandbox mode, what actually happens behind the scenes is that a new data stream is created with a different ID, and the relevant data is re-loaded to the data warehouse with this new ID; all of this is hidden from the\u00a0user.<\/p>\n

    Having both sets of data (Production and Sandbox) reside next to each other in the database with the knowledge of the \u201creplace data\u201d relation between the 2 entities allows us to manipulate the DB query and filter in and out the desired data according to the user\u2019s sandbox\/production mode.<\/p>\n

    For example in Sandbox Mode, the query will filter\u00a0out:<\/p>\n