We are excited to announce that starting on August 3rd, 2024, the Databricks Community will transition to a single sign-on (SSO) login process. This enhancement is designed to streamline your login experience and improve security. Maintenance Schedul...
- 74 Views
- 0 replies
- 2 kudos
yesterday
We are thrilled to announce the launch of our exclusive Certified User Group on the Databricks Community! This group is dedicated to individuals who have achieved Databricks certification and are eager to collaborate, brainstorm, and share insights w...
- 169 Views
- 0 replies
- 1 kudos
yesterday
We are thrilled to announce some fantastic new features that have gone live to enhance your experience in our Community! Your feedback has been invaluable in driving these improvements, and we can't wait for you to check them out. 1. Progress Bar Und...
- 52 Views
- 0 replies
- 2 kudos
Wednesday
Discover the latest Databricks Data Intelligent Platform with our product tours, showcasing the latest innovations in just a few clicks. Join us for an immersive experience where you'll explore the cutting-edge features and capabilities of our platfo...
- 88 Views
- 0 replies
- 2 kudos
Wednesday
Meet G Samhitha Priyadarshini, our featured member for this month! She has been our top contributor and possesses extensive knowledge in Databricks. She is also a Certified Lounge User Group owner. Here are some of the questions we asked her during t...
- 130 Views
- 3 replies
- 2 kudos
yesterday
We are enhancing the Community login process to provide a more secure and streamlined experience for all users. To avoid losing access to your Community account, please update your primary and secondary email addresses now (refer to “Action Required”...
- 3849 Views
- 0 replies
- 4 kudos
06-27-2024
Over 12,000 companies have adopted data intelligence. Join a tour stop near you to discover how Databricks can unlock your data and AI too. Find your city now! Explore technologies Explore the latest advancements in the Databricks Data Intellige...
- 547 Views
- 0 replies
- 4 kudos
a week ago
We are thrilled to introduce DatabricksTV – your go-to hub for all things Databricks! DatabricksTV is packed with insightful videos from a diverse range of creators, offering you the latest tips, tutorials, and deep dives into the Databricks platfor...
- 58 Views
- 0 replies
- 3 kudos
Tuesday
Community Activity
- 7523 Views
- 50 replies
- 6 kudos
FAQ for Databricks Learning Festival (Virtual): 10 July - 24 July 2024
General Q: Where can I ask questions, if I have questions while going through the self-paced material? A: Databricks Academy Learners Group page is the forum within Databricks Community, where you can ask questions and get them answered by your fello...
- 7523 Views
- 50 replies
- 6 kudos
Latest Reply
- 6 kudos
Hello!I received the email with the voucher for the 50% discount, thank you very much. However, the email I received does not list "Databricks Certified Generative AI Engineer Associate", while in the Virtual Festival it did appear that the discount ...
- 6 kudos
- 33 Views
- 1 replies
- 0 kudos
Complex type variable in Databricks.yml not working
For example here I extract the schedule parameter as a complex type variable: variables: schedule: description: schedule time type: complex default: quartz_cron_expression: '0 22 17 * * ?' timezone_id: Asia/Shanghai pa...
- 33 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
It’s strange, it can pass the validation on my colleague’s laptop. I still cannot pass the validation even restart my Terminal and Macbook
- 0 kudos
- 19 Views
- 2 replies
- 1 kudos
Make the job fail if a task fail
Hi everyone,I have a job with 2 tasks running independently. If one of them fails, the remaining task continues to run. I would like the job to fail if any task fails.Is there any way to do that?Thank you!
- 19 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Can you share why you want that to happen? If it's a case of saving on compute, you can repair the run and only rerun the parts that failed. If you want the status code to be a failure, you could use the if/else logic with both tasks as dependents to...
- 1 kudos
- 2425 Views
- 3 replies
- 2 kudos
Passing Parameters Between Nested 'Run Job' Tasks in Databricks Workflows
Posting this on behalf of zaheer.abbas. I'm dealing with a similar scenario as mentioned here where I have jobs composed of tasks that need to pass parameters to each other, but all my tasks are configured as "Run Job" tasks rather than directly runn...
- 2425 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Kaniz_Fatma , are you able to provide clarity on the question above? I'm similarly trying to reference a task value set upstream by a "Run Job" task type in a downstream task. When I pass the task value set in the "Run Job" task to the downstrea...
- 2 kudos
- 31 Views
- 1 replies
- 0 kudos
Cannot Create databricks account
I am trying to create databricks community account but after providing all the information and completing the puzzle it is showing me an error occurred: I also recorded the the network request for the error:Header:Request URL:https://www.databricks.c...
- 31 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello, I can see the error message is after the 'continue' button, what happens instead if you click on the blue 'Get Started with Community Edition' one?
- 0 kudos
- 37 Views
- 1 replies
- 0 kudos
Why all workspace users can see my user folder
Hi, I am a Databricks account admin user with admin access to our workspace. My user folder for some reason is visible to all workspace users. I have checked permissions settings where possible and cannot see anything that would indicate fully shared...
- 37 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello, even with the lowest permissions people will still be able to see / list the folders https://docs.databricks.com/en/security/auth/access-control/index.html#folder-acls
- 0 kudos
- 70 Views
- 2 replies
- 3 kudos
Welcome to all our certified members! 🎉
We're thrilled to have you join our certified community. To kick things off, we’d love to learn more about you. Please start a thread and share the certifications you've received and how you'd like to collaborate with others in this group. Make sure ...
- 70 Views
- 2 replies
- 3 kudos
Latest Reply
- 3 kudos
Hello,I’m Rishabh Pandey, an Associate Consultant - Data Engineer and Technical Lead - COE L&D at Celebal Technologies and Leading Databricks community - Jaipur. I hold certifications as an Associate Data Engineer and Professional Data Engineer from ...
- 3 kudos
- 24 Views
- 1 replies
- 0 kudos
Merge Update in Notebook Faster Than Scala script
Hi Folks,I tested running a merge update using SQL queries in a notebook, and it is faster than using a Scala script. Both tests were done using the same cluster size in Databricks.How can I make the Scala script as fast as the SQL notebook?
- 24 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Have you already compared both query plans?
- 0 kudos
- 25 Views
- 1 replies
- 0 kudos
JDBC Error with OpenJDK 21
Connecting to Azure Databricks using latest JDBC driver and OpenJDK 21 I got this error. I already set:--add-opens=java.base/java.nio=ALL-UNNAMED-Dio.netty.tryReflectionSetAccessible=true java.sql.SQLException: [Databricks][JDBCDriver](500618) Error ...
- 25 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @dprutean, Additionally, you might need to add: --add-opens=jdk.unsupported/sun.misc=ALL-UNNAMED Also, I could see that you have mentioned using the latest JDBC driver. Could you please try an earlier version of OpenJDK, such as OpenJDK 17, which...
- 0 kudos
- 26 Views
- 0 replies
- 0 kudos
SCIM provisioning for user and group management in Databricks. I need to configure the SCIM endpoint
Hi Databricks Community,I am currently working on integrating SCIM provisioning for user and group management in Databricks. I need to configure the SCIM endpoint and token for automated user provisioning.Could you please confirm if there is a specif...
- 26 Views
- 0 replies
- 0 kudos
- 47 Views
- 1 replies
- 0 kudos
Databricks workflow each task cost
Suppose if we have 4 tasks (3 notebooks and 1 normal python code) in a workflow then i would like to know the cost incurred for each task in the Databricks workflow. Please let me know the any way to find out this details.
- 47 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
If each of the tasks are sharing the same cluster then no, you cannot differentiate the costs between the tasks. However, if you setup each task to have its own job cluster, then pass some custom tags and you can then differentiate/report the costs ...
- 0 kudos
- 102 Views
- 3 replies
- 2 kudos
Error Handling and Custom Messages in Workflows
I would like to be able to get custom error's message ideally visible from Workflows > Jobs UI.1. For example, workflow failed because file was missing and could not find it, in this case I am getting "Status" Failed and "Error Code" RunExecutionErro...
- 102 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
What you can do is pass the custom error message you want from the notebook back to the workflow output = f"There was an error with {error_code} : {error_msg}"dbutils.notebook.exit(output) Then when you are fetching the status of your pipeline, you c...
- 2 kudos
- 14 Views
- 0 replies
- 0 kudos
M0tion Energy | Krem | Przedstawiamy | Cena | Recenzja | Efekt | Poland
Two new, more compact sizes of Topical Bio Medics' all-natural M0tion Energy Foot Remedy Krem are now available. The pain relief healing Krem, which was originally packed into an 8-ounce bottle, is now available in a four-ounce jar and a two-ounce tr...
- 14 Views
- 0 replies
- 0 kudos
- 136 Views
- 3 replies
- 3 kudos
From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications
In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...
- 136 Views
- 3 replies
- 3 kudos
Announcements
Latest from our Blog
Eight Weeks, Two Developers, One Production AI Conference Assistant
Introduction: The Origins of Brickbot If you attended the Databricks Data and AI Summit (DAIS) this year, you may have noticed a new addition to the event app: Brickbot, the summit AI assistant. Brick...
You should know these 4 things about setting up Unity Catalog!
Over the past year we’ve seen huge adoption of Unity Catalog, and we've noticed certain considerations that are key to all successful deployments. This blog focuses on how to set up a key artifact in ...
Databricks SSO with Auth0 using SAML 2.0
With the recent changes to Databricks login in order to increase our customers' security posture, some customers might be scrambling to setup SSO on their Databricks account and workspaces. Many of t...
Foundation Models API Prompting Guide 1: Lifecycle of a Prompt
This is the first part of a guide on writing prompts for models accessible via the Databricks Foundation Model API, such as DBRX and Llama 3. Prompting, whether in the context of interacting with a ch...
