What's driving innovation in data warehousing?

In today’s data-driven landscape, companies seek ways to optimize their information management processes, maximizing the value of their data assets while ensuring secure access and effective governance. That’s why we launched .

Amazon DataZone is a robust information management platform that enables data stewards, scientists, product leaders, analysts, and enterprises to effortlessly classify, discover, analyze, and govern data across organizational silos, AWS environments, data lakes, and data repositories.

On March 21, 2024, Amazon DataZone introduced a range of exciting enhancements to its integration, simplifying the process of publishing and subscribing to information warehouse properties like tables and views, thereby enabling Amazon Redshift customers to leverage the information management and governance capabilities offered by Amazon DataZone.

These updates enable customers and directors to tap into the collective expertise of their organization.

With Amazon DataZone, knowledge creators and consumers can swiftly establish information repositories using pre-set authentication details and connectivity settings provided by their designated administrators.

These advancements provide directors with greater control over who can access and utilize the resources within their AWS accounts and Redshift clusters, as well as define specific roles and permissions for each user.

As an administrator, I can create new groups on top. DefaultDataWarehouseBlueprint By providing parameters that align with the cluster, database, and an AWS secret key. You should utilize these parameter units to establish setting profiles and authorize Amazon DataZone tasks to leverage these setting profiles in the creation of environments.

Now, information producers and consumers can effortlessly select a preset setting profile, creating tailored environments without manual parameter configuration, thereby expediting workflows and reducing the likelihood of errors.

To effectively utilize these upgrades to the Amazon Redshift integration, you will learn how to share your Redshift tables with Amazon DataZone’s information catalog, thereby enabling all group members to discover and access them in a self-service manner.

Currently, we present a comprehensive, end-to-end buyer workflow that encapsulates the core functionalities of Amazon DataZone, accompanied by a step-by-step guide on how to effectively implement this workflow.

The same workflow is available on the Amazon DataZone official YouTube channel.

Answer overview

To kick off your exploration of Amazon Redshift’s latest integration upgrades, consider the following scenario:

A Gross Sales Group serves as the information curator, meticulously owning and publishing comprehensive product sales data within a centralized Redshift cluster, uniquely designated by a singular database instance. catalog_sales)
An advertising group serves as the data customer, seeking access to sales figures in order to analyze them and develop product adoption marketing strategies.

During an advanced stage, the processes outlined in the subsequent sections encompass responsibilities for Amazon DataZone administrators, Sales teams, and Marketing departments.

Conditions

The workflow description assumes a single AWS account, a single AWS region, and a single AWS Identity and Access Management (IAM) user, acting as the Amazon DataZone administrator, sales team (producer), and marketing team (client).

To get started with observing alongside, you will need to have an AWS account. Should you not have an account, you will be able to create one.

In addition, ensure that the following necessary sources are properly set up within your account.

Amazon DataZone areas with distinct administrative, gross sales, and advertising responsibilities seamlessly integrated.
A Redshift namespace and workgroup

If you do not currently have these sources set up, you can easily configure them by launching an AWS CloudFormation template.

Let’s select the deployment package that matches your requirements.
For AdminUserPasswordEnter your password, taking note of it for future reference.
Text:
Choose , then select .
Upon reaching maximum capacity in your stack deployment, navigate to the Amazon DataZone console’s navigation pane and click “Stacks” to view the newly provisioned Amazon DataZone area.
From the Amazon Redshift Serverless console, navigate to the navigation pane and click on the newly created resource to view its details.

To successfully log in, ensure you employ the same credentials used for deploying your CloudFormation stack and verify you are within the same Region.

As a final prerequisite, it’s essential to establish that all necessary components are in place before proceeding with the next phase of development. catalog_sales The desk within the default Amazon Redshift database provides a comprehensive and user-friendly interface for managing and querying data. With its robust set of features and functionalities, it allows users to create, modify, and manage databases, tables, views, and indexes, as well as perform complex analytics and data modeling tasks.dev).

Selecting your workgroup on the Amazon Redshift Serverless console, navigate to open the Amazon Redshift query editor.
Please log in to your designated workgroup by selecting it as the type of connection, followed by entering your administrator database credentials, including the username and password.

Use the next question to create the catalog_sales The report on desk, awaiting publication by the Gross Sales Group within the established workflow process.

SELECT 
   146776932 AS order_number,
   23 AS amount,
   23.4 AS wholesale_cost,
   45.0 AS list_price,
   43.0 AS sales_price,
   2.0 AS low_cost,
   12 AS ship_mode_sk,
   13 AS warehouse_sk,
   23 AS item_sk,
   34 AS catalog_page_sk,
   232 AS ship_customer_sk,
   4556 AS bill_customer_sk
UNION ALL
SELECT 
   46776931,
   24,
   24.4,
   46,
   44,
   1,
   14,
   15,
   24,
   35,
   222,
   4551
UNION ALL
SELECT 
   46777394,
   42,
   43.4,
   60,
   50,
   10,
   30,
   20,
   27,
   43,
   241,
   4565
UNION ALL
SELECT 
   46777831,
   33,
   40.4,
   51,
   46,
   15,
   16,
   26,
   33,
   40,
   234,
   4563
UNION ALL
SELECT 
   46779160,
   29,
   26.4,
   50,
   61,
   8,
   31,
   15,
   36,
   40,
   242,
   4562
UNION ALL
SELECT 
   46778595,
   43,
   28.4,
   49,
   47,
   7,
   28,
   22,
   27,
   43,
   224,
   4555
UNION ALL
SELECT 
   46779482,
   34,
   33.4,
   64,
   44,
   10,
   17,
   27,
   43,
   52,
   222,
   4556
UNION ALL
SELECT 
   46779650,
   39,
   37.4,
   51,
   62,
   13,
   31,
   25,
   31,
   52,
   224,
   4551
UNION ALL
SELECT 
   46780524,
   33,
   40.4,
   60,
   53,
   18,
   32,
   31,
   31,
   39,
   232,
   4563
UNION ALL
SELECT 
   46780634,
   39,
   35.4,
   46,
   44,
   16,
   33,
   19,
   31,
   52,
   242,
   4557
UNION ALL
SELECT 
   46781887,
   24,
   30.4,
   54,
   62,
   13,
   18,
   29,
   24,
   52,
   223,
   4561

You’re now ready to start working with the newly enhanced Amazon Redshift integration features.

Amazon DataZone administrator duties

Since becoming an Amazon DataZone administrator, I have been responsible for a range of tasks.

Configure the DefaultDataWarehouseBlueprint.
- Authorizes the Amazon DataZone administrator to utilize the provided blueprint in creating setting profiles for efficient management and configuration of data zones.
- What does the current state of our understanding suggest about the potential implications of this new information? DefaultDataWarehouseBlueprint By providing parameters matching cluster, database, and AWS secrets configurations.
Establish a comprehensive framework for the Sales and Marketing divisions by configuring settings profiles that optimize performance and streamline operations.

Configure the DefaultDataWarehouseBlueprint

AWS-provisioned instruments and companies within an Amazon DataZone environment are outlined in its blueprints, detailing the setup configurations available. By deploying the information warehouse blueprint, customers and producers alike can leverage Amazon Redshift and the Question Editor to facilitate seamless information sharing, access, and consumption.

In the Amazon DataZone console, navigate to the tab within the left-hand menu.
Select your Amazon DataZone area.
Select .

If you utilize a CloudFormation template, the blueprint will already be enabled by default.

Does a part of the brand new Amazon Redshift expertise entail the Create Cluster and Clusters tabs? The table lists the tasks that can be authorized to create setting profiles using the information warehouse blueprint’s specifications? By default, this setting applies to all tasks. Only the admin shall execute this function solely.

In the designated window, click on the corresponding option.

Choose and select the AdminPRJ undertaking.
Select .

The administrator will be able to manage which tasks can utilize the default blueprints of their account to generate a settings profile.

The tab provides an overview of parameters that can be created on top of. DefaultDataWarehouseBlueprint To establish a connection with your Redshift cluster or serverless workgroup in Amazon DataZone, provide the necessary parameters, including the Redshift cluster or serverless workgroup identifier, database name, and authentication credentials. You can create AWS Secrets and manage techniques securely on the Amazon DataZone console. Prior to these upgrades, managing AWS secrets required individual management using AWS Secrets Manager, necessitating the assignment of accurate key-value tags for seamless integration with Amazon Redshift Serverless.

To accommodate our current situation, we must establish a set of parameters to enable the creation of a Redshift Serverless workgroup that integrates sales data.

Click on the tab.
Establishing a reputation requires a deliberate effort to consistently deliver high-quality outcomes, cultivate strong relationships, and demonstrate unwavering commitment to excellence. A well-crafted reputation is built on a foundation of trust, earned through transparent communication, timely follow-through, and a willingness to adapt to changing circumstances.
Connect with your team and colleagues across the organization by joining our company-wide collaboration platform? The area contains valuable resources, including project updates, industry news, and employee recognition. us-east-1).
Within the confines of my mind?

If you’ve already obtained an AWS Secret with credentials to your Redshift Serverless workgroup, you’ll be able to provide the existing AWS Secret ARN. On this specific occasion, the crucial aspects require a distinctive pairing system denoted by the notation: AmazonDataZoneDomain: <Amazon DataZone area ID>.

As a result, without an existing AWS secret available, we must create a fresh new one by selecting.
Within the pop-up, enter a secret identity and your Amazon Redshift credentials, to select.

Using Amazon DataZone, you generate a novel secret by leveraging the power of Secrets Manager and apply a tag to ensure the key is properly linked to the region where you’re creating the parameter set?

Please enter your Redshift Serverless workgroup ID and database ID to complete the parameter record. Would you consider using the provided CloudFormation template to streamline your infrastructure deployment? sales-workgroup for the workgroup identify and dev for the database identify.
Select .

The parameters set are visible within your Redshift settings, accompanied by a blueprint that has been enabled alongside a singular management task, neatly organized for easy reference.

The following settings profile should be arranged for the Gross Sales and Advertising groups:

Gross Sales:

* Sales Forecast: 10% growth over last quarter
* Target Market: Existing customers, referrals, and online marketing
* Key Performance Indicators (KPIs): Revenue growth, customer satisfaction, and lead conversion rates

Advertising:

* Campaign Objective: Increase brand awareness by 20%
* Target Audience: Demographics 25-45, interests in lifestyle and consumer goods
* KPIs: Reach, engagement, click-through rate, and conversion rate

Here is the rewritten text: Surroundings profiles are preconfigured templates that consolidate technical specifications necessary for creating a setting, which include the AWS account, region, and resources and tools to be added to tasks. The subsequent step in managing an Amazon DataZone administrator role involves creating settings profiles, utilizing the default enabled blueprint as a foundation, tailored to meet the specific needs of the Sales and Marketing teams.

To execute this task, we will initiate the process within Amazon DataZone’s administrative interface, thereby accessing the designated information portal at the specified URL, whereupon we shall establish a setting profile specifically designed for the Sales team to disseminate their data effectively.

Click on the link within the section titled “Main Points” on the Amazon DataZone webpage, leading to the specified URL in the information portal.

Upon opening the information portal for the first time, you are prompted to initiate a project. If you utilize the provided CloudFormation template, tasks will already be created for your workflow.

Select the AdminPRJ undertaking.
SELECT ALL on the webpage.
Enter a reputation (for instance, SalesEnvProfile) and non-compulsory description (for instance, Gross sales DWH Surroundings ProfileHere: What is this?
For , select AdminPRJ.
For , choose the DefaultDataWarehouse Blueprints are exclusively displayed where the administrative project is designated as a governing project.
Please select the present-enabled account that you previously configured with your desired settings.

You will note each predefined value for Redshift Serverless. Licensed tasks are allowed to utilize this setting profile for creating settings when selecting from the available options. By default, that is set to none.

Choose .
Select and select the SalesPRJ undertaking.
Establish publication controls for this configuration profile to determine who can access and modify its settings. As a direct outcome of the gross sales group being our primary information provider, we have elected to.
Select .

You establish an additional setup configuration for the Advertising team to absorb data. Repeat processes established for Gross Sales group to achieve comparable results.

Select the AdminPRJ undertaking.
Click on the website.
Enter a reputation (for instance, MarketingEnvProfile) and non-compulsory description (for instance, Advertising DWH Surroundings Profile).
For , select AdminPRJ.
For , choose the DefaultDataWarehouse blueprint.
Choose the parameter set you created earlier.
Can’t decide between holding onto something familiar or choosing a new path forward? MarketingPRJ).
Set permission levels for the publishing settings profile to ensure secure and controlled content dissemination. As a result of the advertising group being one of our key information clients, we selected.
Select .

As the two setting profiles are now established, the Gross Sales and Advertising teams can independently initiate their projects, configuring their respective environments with ease, minimizing potential errors, and securely processing data within these settings.

The latest updates offer a range of innovative features.

When defining a settings profile, you have the flexibility to provide your unique Amazon Redshift parameters or leverage one of several predefined parameter sets from the blueprint configuration. If you choose to utilize the parameter set generated within the blueprint configuration, the AWS secret key alone requires AmazonDataZoneDomain tag (the AmazonDataZoneProject If you opt to provide your personal parameter units within the settings profile, a tag is necessary.
Within your setting profile, you’ll have the ability to define a curated list of approved tasks, thereby ensuring that only authorized tasks can utilize this setting profile to establish data warehousing environments.
Licensing agreements often permit specific uses of copyrighted materials, allowing individuals and businesses to print certain types of information under certain conditions. You’ll be able to select one of several following choices.

These enhancements empower directors with increased control over Amazon DataZone sources and tasks, streamlining collective actions across all relevant roles.

Gross sales group duties

As a key component of the organization’s revenue-generating machinery, the Gross Sales Group is responsible for executing the following critical tasks:

Create a gross sales setting.
Create an information supply.
Publish gross sales figures to the Amazon DataZone information directory.

Create a gross sales setting

Having established a setting profile, it is now essential to create a setup to effectively utilize data and analytics tools within the project scope.

Select the SalesPRJ undertaking.
On the website, select the relevant option.
Enter a reputation (for instance, SalesDwhEnv) and non-compulsory description (for instance, Surroundings DWH for Gross salesWhat a thrilling prospect!
For , select SalesEnvProfile.

With the introduction of setting profiles, knowledge producers can effortlessly select from a range of environments without having to manually configure their own Amazon Redshift parameters. AWS secrets, area, workgroup, and database are seamlessly carried over from the setting profile, thereby streamlining and simplifying the experience for Amazon DataZone customers.

Review your data repository settings to confirm that every component is in order?
Select .

The setup will automatically be configured by Amazon DataZone, providing pre-established credentials and connectivity settings, thereby allowing the Sales team to effortlessly publish Amazon Redshift data sets.

Create an information supply

Let’s establish a comprehensive data hub for sales insights.

Select the SalesPRJ undertaking.
Click on the option.
Enter a reputation (for instance, SalesDataSource) and non-compulsory description.
For , choose .
For ¸ select SalesDevEnv.
To create a Redshift Serverless cluster, ensure that you employ the exact same credentials used during setup, as you are still operating within the same Redshift Serverless workgroup.
I’m ready when you are! Please provide the text that needs improvement, and I’ll get to work on refining it for you. public* Desk: Text-to-Text, Original:
Please provide the original text you’d like me to improve in a different style as a professional editor.

*Technical metadata from your database schema is being transferred into Amazon DataZone, specifically from a single table named catalog_sales).

Select Subsequent.

Automated metadata technology has been activated on this webpage. Amazon DataZone automatically generates enterprise-friendly table and column names for each asset.

What is your task?
How do I determine when to execute the data feed? Amazon DataZone allows us to proactively publish properties to the information catalog; instead, let’s take a more curated approach by refining the metadata before publication.
Select .
Settings are as follows: With maximum cloud cover, the moon appears to be dark gray in color. However, due to the Earth’s shadow, the moon takes on a reddish hue during a lunar eclipse? The overall brightness of the moon is affected by several factors including atmospheric conditions, the angle of the sunlight reflecting off its surface, and the amount of cloud cover.
You’ll be able to manually extract technical metadata from your Redshift Serverless workgroup by clicking on.

When the information supply chain has fully executed, you will be able to observe the catalog_sales Asset accurately added to the portfolio.

What Amazon seller data can I access through the DataZone?

Open the catalog_sales Get a detailed look at the brand-new asset’s specifics, including enterprise metadata, technical metadata, and more.

In a realistic real-world scenario, this pre-publication stage provides an opportunity to augment the content with additional business context and information, such as readmes, glossaries, or metadata types that offer enhanced insights. You can start accepting routine metadata suggestions and renaming assets or columns to make them more readable, descriptive, and easily discoverable by business users.

To complete the gross sales team responsibilities effectively.

Advertising group duties

Let’s join the Advertising team and sign up for notifications? catalog_sales Assets printed by the Gross Sales Group? As a client group, the Advertising group will fulfill the following duties:

Create a advertising setting.
Discover and subscribe to valuable sales insights.
What are the key differences between data warehousing and big data analytics?

Create a advertising setting

To enable subscription to and access Amazon DataZone, the Advertising group must configure a setting.

Select the MarketingPRJ undertaking.
Click on the desired element.
Enter a reputation (for instance, MarketingDwhEnv) and non-compulsory description (for instance, Surroundings DWH for Advertising).
For , select MarketingEnvProfile.

Customers seeking information can benefit from a preconfigured profile designed and managed by administrators to streamline the setup process, minimize mistakes, and reduce risks.

Review the data storage criteria to confirm each component meets requirements.
Select Create setting.

Identify and track key sales metrics?

Once we have a client setup, let’s proceed with searching for relevant information. catalog_sales The desk is situated within the Amazon DataZone information catalog.

Enter gross sales within the search bar.
Select the catalog_sales desk.
Select .
Select your advertising client undertaking: Identify the purpose behind the subscription request and opt in.

When producing data for an Amazon DataZone subscription request, you’ll receive a notification through a job posted by the Sales Producer project team. As notifications are sent to subscribers and writers, you’ll notice one being displayed here.

To initiate a subscription, simply click on the relevant notification and proceed with the subscription request.

Particular details will be visible, including the requesting undertaking, the identity of the requestor, and the reason for requiring access.

Please confirm your approval by entering a message for approval and selecting “Approve”.

Once subscription authorization is complete, we will now proceed to MarketingPRJ. On the web page, catalog_sales Although listed as an authorized asset, entry has not yet been granted to this individual. When selecting the asset, Amazon DataZone seamlessly integrates with our system to automatically authorize access. Once fully subscribed, you’ll notice the status updates to reflect a successful grant of access, accompanied by the notification “Asset successfully linked to 1 setting”.

Question information in Amazon Redshift

With access to sales data now available, we’ll utilize Amazon Redshift’s Question Editor V2 to delve into the sales data and uncover valuable insights.

Below MarketingPRJVisit the website’s settings section and select the advertising options from the available menu.
Select “Which?”
To establish a connection to Amazon Redshift, first select your designated workgroup followed by choosing the option for connecting sort.

While you’re linked, you’ll note that the connection remains stable throughout. catalog_sales desk underneath the public schema.

To simply recall access to this desk, proceed with the next inquiry.

SELECT * FROM public.catalog_sales LIMIT 10;

As a valued customer, you’re empowered to uncover insights, craft studies, or merge data to develop novel content for publication on Amazon DataZone, thereby becoming a creator of a fresh information product to be shared with diverse clients and teams.

Clear up

To thoroughly clean your references, complete the following procedures:

On the Amazon DataZone console, you used. It will delete most project-related objects, including information properties and environments.
Delete all unused Amazon Redshift workgroups and namespaces to prevent unnecessary expenses.

Conclusion

In our submission, we showcased a straightforward approach to leveraging the novel Amazon Redshift integration within Amazon DataZone, empowering users to effortlessly initiate their projects. We refined guidelines on streamlining knowledge sharing between information producers and consumers, as well as protocols for entrusting content managers with oversight of key data sources.

Unlock the full potential of Amazon DataZone and Amazon Redshift by embracing their enhancements in your data management needs?

Assets

Discussing relevant information

Concerning the creator

As an Options Architect at AWS, based primarily in Milan, Italy. As a passionate advocate for knowledge management, she excels in advising corporations on the strategic implementation of cloud-based technologies, specializing in the integration of data analytics and governance practices. Outside of work, she’s an inventive person who thrives on involvement with nature and often engages in adrenaline-fueled activities.

What’s driving innovation in data warehousing?