AWS wants every data silo on its platform • The Register

RE: INVENT AWS introduced a host of new database and ML services at its Re: invent conference, including a migration service targeting every database in an organization,

ML (Machine Learning) Vice President Swami Sivasubramanian gave the keynote address on data today. He claimed that Aurora, a MySQL or PostgreSQL compatible service, has “5 times the performance of MySQL and 3 times the performance of PostgreSQL” and “is still the fastest growing service in AWS history. “.

Nonetheless, he was concerned that “some customers are being prevented from migrating to databases in the cloud”. It introduced two new services aimed at getting an even larger proportion of an organization’s databases to the AWS Cloud.

The first is Custom Relational Database Service (RDS) for Microsoft SQL Server. This joins the existing RDS Custom for Oracle, and both services are intended for customers with “mission-critical applications such as the Oracle E-Business suite, or Microsoft Dynamics, or SharePoint, which were designed to run on commercial databases. with very specific configurations ”.

RDS Custom is a hybrid between RDS, where the customer has no direct access to the operating system, and running SQL Server or Oracle on EC2, where every detail is the customer’s responsibility. RDS Custom allows full privileged access to the operating system, allowing the installation of custom apps and agents, while providing some of the benefits of RDS, such as automated point-in-time recovery and monitoring. state. Ideal, said Sivasubramanian, for applications like SharePoint, Dynamics, Pow

erBI and Polybase.

Next, Sivasubramanian said that “customers have told us that developing a migration plan for their entire database fleet is difficult,” the assumption being that organizations want their entire database. on AWS if they can only figure out how. “Today we are previewing AWS DMS Fleet Advisor,” he said, where DMS is the database migration service.

Fleet advisor is “a new feature in AWS DMS that automates migration planning for an entire database fleet. DMS Fleet Advisor automatically creates an inventory of your on-premises database and analytics servers by streaming the on-premises data to Amazon S3. We analyze them to match them with the correct AWS data store and customize the migration plans. It only takes hours now, ”he promised. “You don’t have to rely on a third-party tool or an expensive migration consultant. “

Trust the agent

Database administrators may be skeptical, but the idea is that users install an agent on their networks, in one or more locations, which collects data and builds an inventory, uploading its results to S3.

From the resulting inventory, “you can convert schemas for migration using the AWS Schema Conversion Tool,” the docs say. This has many options, including for example converting from SQL Server to Aurora MySQL; or the databases could simply be migrated to the same database manager running on AWS.

Inventories discovered by DMS Fleet Advisor

Inventories discovered by DMS Fleet Advisor

Typically, these tools turn out to have limitations, but there is no doubt that the company is committed to delivering all varieties of databases on its ever-expanding platform. “Over 500,000 databases have been migrated to AWS with AWS Database Migration Service,” said Sivasubramanian.

Another statistic mentioned is that there are over 200,000 data lakes on AWS. Sivasubramanian highlighted the number of different types of databases that AWS offers, including DocumentDB for key-value pairs, Neptune for Graph data, TimeStream for event processing, and ElastiCache for in-memory caching.

There are a ridiculous number of new features announced here at Re: invent, and the Data and ML category is no exception. DevOps Guru for RDS is a new service for detecting and analyzing issues related to the performance or operation of Aurora. DynamoDB gets a new infrequently accessed table class which is claims, could reduce costs by up to 60% for tables that store infrequently accessed data. “

SageMaker Canvas, a drag-and-drop user interface for creating ML models

SageMaker Canvas, a drag-and-drop user interface for creating ML models

On the ML side, SageMaker, an ML model development tool, has a new training compiler that could double performance through better use of GPU instances; and SageMaker Studio Canvas, a code-free drag-and-drop environment for creating SageMaker models, which Sivasubramanian has positioned as an alternative to finding predictions using numbers in spreadsheets. There is also SageMaker Serverless Inference, which allows users to deploy models without having to provision compute resources.

Cryptocurrency mining is prohibited

SageMaker Studio Lab is a free ML service base on the open source JupiterLab, limited to one project and up to 15 GB of storage. Users can select a CPU or GPU runtime and use it up to 12 hours (CPU) or 4 hours (GPU). Files are kept although runtimes are closed after each session. Availability is not guaranteed, and “Cryptocurrency mining is prohibited.” Support is community only.

A service called SageMaker Ground Truth Plus involves real people, with AWS promising to hire experts to label a customer’s data to create a model. “For example, if you need medical experts to label X-ray images, you can specify that in the guidelines you provide to Ground Truth Plus. The service will then automatically select radiologically trained labelers to label your data, ”said the post today.

How many? As you might expect, after applying, “our team of AWS experts will schedule a call to discuss your data labeling project.” The existing Ground Truth service Free the ability to use “Mechanical Turk, third-party vendors, or your own private workforce” to label the data, but in the new service, AWS itself is taking responsibility for this critical task.

Sivasubramanian also introduced a new feature for Kendra, the AWS enterprise search service. Apparently, customers struggled to create a UI for Kendra, so now the experience builder will allow UI to be created with just a few clicks and no code.

In addition, Search Analytics provides data on how Kendra is used, and Custom Document Enrichment preprocesses documents with automatic classification, image-to-text conversion, and additional metadata using custom processes. Making it easy to move databases to AWS is good business for the business. The innovation, however, lies in the area of ​​ML accessibility, up to the provision of experts to label the data. ®

Maria H. Underwood