Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,841 questions
0 answers
getting CLUSTER_CREATION_TIMED_OUT issue
Hi, my jobs are failing without spark session is starting and getting timed out issue. all my pools are XXL configuration but inside pipeline calling Small or medium.
asked 2024-09-06T22:45:27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 99%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Venkateswarlu Pottapalli
0
Reputation points
asked 2024-09-06T22:45:27+00:00
Venkateswarlu Pottapalli
0
Reputation points
3 answers
One of the answers was accepted by the question author.
ADF REST Linked Services Auth Headers are not working
This stackoverflow thread has a good explanation of the same issue I am seeing: https://stackoverflow.com/questions/78948962/adf-rest-linkedservice-is-missing-authheader-in-live-mode-after-publishing/78950885#78950885 I need auth headers to be entered…
asked 2024-09-04T23:48:49.1666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJL%3C/text%3E%3C/svg%3E)
Joseph Lee
25
Reputation points
commented 2024-09-06T20:53:22.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
30,496
Reputation points Microsoft Employee
0 answers
Moving data from Snowflake to Azure SQL DB using Copy Activity
Hi All, I am using COPY activity to move data from Snowflake to Azure SQL DB. First of all, this data movement is not allowed without intermediate staging of the data in ADLS. I would love to avoid this intermediate step. If this is how it will work then…
asked 2024-09-04T14:26:06.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 27%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
PK
0
Reputation points
commented 2024-09-06T19:08:04.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
14,300
Reputation points Microsoft Vendor
1 answer
Fabric Data fetching API
I have created an azure app from which I need to pull user data present on their fabric workspaces (lakehouses, warehouses etc.). Fabric APIs like : GET…
asked 2024-09-05T14:50:50.1266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 63%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Shubham
0
Reputation points
commented 2024-09-06T19:02:12.3666667+00:00
Smaran Thoomu
14,300
Reputation points Microsoft Vendor
0 answers
Exporting data via an OData API
I am experiencing issues while attempting to retrieve data from multiple Cornerstone OnDemand (C SOD) API views using Azure Data Factory. The process works seamlessly on Postman, but when implemented in ADF, I encounter authentication and request…
asked 2024-09-03T11:07:57.1066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 17%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
A K
0
Reputation points
commented 2024-09-06T19:00:50.0466667+00:00
Smaran Thoomu
14,300
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Preserving Nested Zip Structure in ADF Copy Activity
I'm working with Azure Data Factory (ADF) and using a Copy Data activity to unzip compressed files in Blob storage. It functions flawlessly for simple text files within basic zip archives. However, I encounter an issue when dealing with nested zip…
asked 2024-09-04T18:35:48.8033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV0%3C/text%3E%3C/svg%3E)
vikranth-0706
265
Reputation points
accepted 2024-09-06T17:47:24.3933333+00:00
vikranth-0706
265
Reputation points
0 answers
Azure Data Factory - Cannot open nested pipelines by double clicking on them.
Hi Everyone, My apologies if this is not logged/asked in the correct place, please point me to the correct location if needed. I am having a strange but frustrating issue with data factory. When working on pipelines, specifically nested pipelines, I…
asked 2024-09-06T08:28:15.0466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 64%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHV%3C/text%3E%3C/svg%3E)
Herman van Eeden
1
Reputation point
commented 2024-09-06T17:26:59.34+00:00
Smaran Thoomu
14,300
Reputation points Microsoft Vendor
0 answers
Integration runtimes are not auto-updating
Our Azure Data Factory integration runtimes are not auto-updating even though we have them set to auto-update. We checked the troubleshooting items listed, and none seem to apply. auto-update is turned on we are able to access the download center…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 47%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
2 answers
Error when using the CosmosDB Analytical Store for source queries in a Synapse ADF Dataflow
How can I resolve the error message I receive when using the CosmosDB Analytical Store as source queries in a Synapse ADF Dataflow? I created the source with Analytical Store selected as explained in this how-to link:…
asked 2024-09-06T10:59:31.9366667+00:00
Jude Moore
0
Reputation points
answered 2024-09-06T16:11:00.35+00:00
Amira Bedhiafi
22,616
Reputation points
2 answers
One of the answers was accepted by the question author.
Role required to read/write data from cosmos DB (SQL API) from ADF?
We are trying to read/write data from Azure data factory, since local authentication is disabled in cosmos we are trying to access cosmos DB using managed identity. However even with Cosmos DB Account Contributor role assigned to managed identity of…
asked 2022-04-06T15:47:26.763+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 92%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ashutosh Saini
36
Reputation points
commented 2024-09-06T15:55:17.5433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 88%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGB%3C/text%3E%3C/svg%3E)
Guillaume Belzile
0
Reputation points
1 answer
Dynamic Mapping of Source to JSON Sink in Azure Data Factory
We are able to preview the source data from the Salesforce linked service in ADF, and I can also view the sink data using the REST linked service for Business Central in ADF. However, I'm facing an issue where importing the schema is not supported…
asked 2024-09-06T12:01:43.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 4%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Apoorva Belsare
0
Reputation points
answered 2024-09-06T15:53:19.94+00:00
Amira Bedhiafi
22,616
Reputation points
1 answer
Google BigQuery updated linked service error: "Message=No current element is available" for empty result
I started to receive failure with error "Message=No current element is available" for Copy activity with the new Google BigQuery linked service. The legacy one doesnt throw an error/failure when the Google SQL query gives an empty result. I…
asked 2024-07-01T14:42:56.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 0%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQD%3C/text%3E%3C/svg%3E)
Quyen Dang
21
Reputation points
commented 2024-09-06T15:27:35.3+00:00
Bhargava-MSFT
30,496
Reputation points Microsoft Employee
1 answer
ADF Snowflake Connector - Key Pair Authentication - Failing Using Self-Hosted IR
I have created a Linked Service using the SnowflakeV2 connector. The connector is configured to use Key Pair authentication and using the AutoResolveIntegrationRuntime works perfectly. When switching to a self-hosted integration runtime, I receive the…
asked 2024-09-04T13:54:22.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 51%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Andy Reed
0
Reputation points
commented 2024-09-06T15:22:06.51+00:00
Bhargava-MSFT
30,496
Reputation points Microsoft Employee
1 answer
Pipeline status = success but no data was written to the sink
Pipeline Run ID: 844b967c-aeb1-4dd7-9ba2-14b7a34d85cc Hi, I am trying to debug my ADF Pipeline and cannot see useful details. Can you please help me debug this pipeline? Specifically, I have 4 objects in the pipeline that are looped through. Three of…
asked 2024-09-05T23:37:55.33+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 21%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMJ%3C/text%3E%3C/svg%3E)
Mick Jaeger
0
Reputation points
commented 2024-09-06T15:15:49.4733333+00:00
Mick Jaeger
0
Reputation points
1 answer
Dynamic Mapping of JSON Sink Data in ADF
Hello Microsoft Community, I hope you're all doing well. I'm currently working on a project in Azure Data Factory and need some advice! We are successfully able to preview source data from the Salesforce Linked Service and sink data using the REST…
asked 2024-09-06T12:06:58.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 13%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVJ%3C/text%3E%3C/svg%3E)
Vinit Joshi
0
Reputation points
edited the question 2024-09-06T14:58:22.7366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 73%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
anashetty
0
Reputation points Microsoft Vendor
1 answer
Variable from one environment to another
Hi how to add environment variable when same pipeline needs to load from qa to prod or test instance like link services.. Pls share screenshot
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
2 answers
Error Connecting ADF to Snowflake
Hello. I am tying to connect Azure Data Factory to Snowflake, but I cannot seem to get things configured correctly. I created a Linked Service, then created a Dataset based on that linked service. However, ADF can't seem to return a list of tables: I…
asked 2024-03-11T15:13:34.1866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 66%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Scott Klein
161
Reputation points
edited an answer 2024-09-06T14:43:20.2833333+00:00
Monique ROLLIER
0
Reputation points
1 answer
Unable to connect the ADLS to snowflake
I am not able to connect the ADLS to snowflake. I requested azure to create the connection but I don't see the user that starts with Snowflake... when I create a role in IAM.
asked 2024-09-05T18:47:27.9966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Purnima Sharma
0
Reputation points
answered 2024-09-06T14:20:41.9866667+00:00
Purnima Sharma
0
Reputation points
2 answers
Error while reading vault secret from ADF - Operation returned an invalid status code 'Forbidden'
Hi All, I have a setup a keyVault with RBAC The keyvault is configured to allow access from specific network. Private end point is configured as well and approved. Have provided KeyVault secret user role to the ADF system managed identity. When…
asked 2024-08-17T15:13:58.1733333+00:00
Rupayan Biswas
0
Reputation points
commented 2024-09-06T14:14:44.3466667+00:00
Smaran Thoomu
14,300
Reputation points Microsoft Vendor
1 answer
How do I get the current date in an ADF Data Flow Sink action filename?
This is in Azure Data Factory. I have a data flow which has a sink to output to a .CSV file. I'd like to make the filename have the date like this: "out_20240906.csv". On the Sink action there is a "Filename" field where I can enter…
asked 2024-09-06T12:17:32.2633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 77%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Chuck Roberts
85
Reputation points
answered 2024-09-06T13:05:45.1733333+00:00
Amira Bedhiafi
22,616
Reputation points