| Document Type
|
:
|
BL
|
| Record Number
|
:
|
851206
|
| Main Entry
|
:
|
Rawat, Sudhir.
|
| Title & Author
|
:
|
Understanding Azure Data Factory : : operationalizing big data and advanced analytics solutions /\ Sudhir Rawat, Abhishek Narain.
|
| Publication Statement
|
:
|
[New York, New York] :: Apress,, [2019]
|
| Page. NO
|
:
|
1 online resource (376 pages)
|
| ISBN
|
:
|
1484241223
|
|
|
:
|
: 1484241231
|
|
|
:
|
: 1484247647
|
|
|
:
|
: 9781484241226
|
|
|
:
|
: 9781484241233
|
|
|
:
|
: 9781484247648
|
|
|
:
|
1484241215
|
|
|
:
|
9781484241219
|
| Contents
|
:
|
Intro; Table of Contents; About the Authors; About the Technical Reviewer; Introduction; Chapter 1: Introduction to Data Analytics; What Is Big Data?; Why Big Data?; Big Data Analytics on Microsoft Azure; What Is Azure Data Factory?; High-Level ADF Concepts; Activity; Pipeline; Datasets; Linked Service; Integration Runtime; When to Use ADF?; Why ADF?; Summary; Chapter 2: Introduction to Azure Data Factory; Azure Data Factory v1 vs. Azure Data Factory v2; Data Integration with Azure Data Factory; Architecture; Concepts; Pipelines; Activities; Execution Activities (Copy and Data Transform)
|
|
|
:
|
Activity PolicyControl; Activity Dependency; Datasets; Dataset Structure; When to Specify a Dataset Structure?; Linked Services; Linked Service Example; Integration Runtime; Azure IR; Self-Hosted IR; Azure-SSIS IR; Hands-on: Creating a Data Factory Instance Using a User Interface; Prerequisites; Steps; Hands-on: Creating a Data Factory Instance Using PowerShell; Prerequisites; Log In to PowerShell; Create a Data Factory; Summary; Chapter 3: Data Movement; Overview; How Does the Copy Activity Work?; Supported Connectors; Configurations; Supported File and Compression Formats
|
|
|
:
|
Chapter 7: SecurityOverview; Cloud Scenario; Securing the Data Credentials; Data Encryption in Transit; Data Encryption at Rest; Hybrid Scenario; On-Premise Data Store Credentials; Encryption in Transit; Considerations for Selecting Express Route or VPN; Firewall Configurations and IP Whitelisting for Self-Hosted Integration Runtime Functionality; IP Configurations and Whitelisting in Data Stores; Proxy Server Considerations; Storing Credentials in Azure Key Vault; Prerequisites; Steps; Using the Authoring UI; Reference Secret Stored in Key Vault; Using the Authoring UI
|
|
|
:
|
Copy Activity PropertiesProperty Details; How to Create a Copy Activity; Schema Capture and Automatic Mapping in Copy Data Tool; Scenario: Creating a Copy Activity Using the Copy Data Tool (Binary Copy); Copy Performance Considerations; Data Integration Units; Parallel Copy; Staged Copy; How Staged Copy Works; Configuration; Staged Copy Billing Impact; Considerations for the Self-Hosted Integration Runtime; Considerations for Serialization and Deserialization; Considerations for Compression; Considerations for Column Mapping; Summary; Chapter 4: Data Transformation: Part 1
|
|
|
:
|
Data TransformationHDInsight; Hive Activity; Pig Activity; MapReduce Activity; Streaming Activity; Spark Activity; Azure Machine Learning; Azure Data Lake; Chapter 5: Data Transformation: Part 2; Data Warehouse to Modern Data Warehouse; ETL vs. ELT; Azure Databricks; Build and Implement Use Case; Stored Procedure; Custom Activity; Chapter 6: Managing Flow; Why Managing Flow Is Important; Expressions; Functions; Activities; Let's Build the Flow; Build the Source Database; Build Azure Blob Storage as the Destination; Build the Azure Logic App; Build the Azure Data Factory Pipeline; Summary
|
| Abstract
|
:
|
Improve your analytics and data platform to solve major challenges, including operationalizing big data and advanced analytics workloads on Azure. You will learn how to monitor complex pipelines, set alerts, and extend your organization's custom monitoring requirements. This book starts with an overview of the Azure Data Factory as a hybrid ETL/ELT orchestration service on Azure. The book then dives into data movement and the connectivity capability of Azure Data Factory. You will learn about the support for hybrid data integration from disparate sources such as on-premise, cloud, or from SaaS applications. Detailed guidance is provided on how to transform data and on control flow. Demonstration of operationalizing the pipelines and ETL with SSIS is included. You will know how to leverage Azure Data Factory to run existing SSIS packages. As you advance through the book, you will wrap up by learning how to create a single pane for end-to-end monitoring, which is a key skill in building advanced analytics and big data pipelines.
|
| Subject
|
:
|
Big data.
|
| Subject
|
:
|
Quantitative research.
|
| Subject
|
:
|
Big data.
|
| Subject
|
:
|
Quantitative research.
|
| Subject
|
:
|
Microsoft Azure SQL Database.
|
|
|
:
|
Microsoft Azure SQL Database.
|
| Dewey Classification
|
:
|
005.7
|
| LC Classification
|
:
|
QA76.9.B45
|
| Added Entry
|
:
|
Narain, Abhishek
|