Filter by type or platform. New posts arrive every morning.
Build control flow inside Databricks Jobs instead of an external orchestrator. Pass values between tasks, branch with if/else condition tasks, fan out with for each tasks and concurrency limits, use run-if rules for cleanup, and define it all in an asset bundle.
Improve recall on vague, conversational and multi-part questions by transforming the query before retrieval. Implement standalone-question rewriting, multi-query with reciprocal rank fusion, HyDE and decomposition, then prove the gain with a retrieval evaluation set and keep latency in budget.
Replace copy-pasted tasks and sensor chains with dynamic task mapping and dataset (asset) triggered DAGs. Control fan-out with max_active_tis_per_dag and pools, handle partial failures, and avoid the mapping patterns that overload the scheduler.
Write a small, correct Airflow DAG with the TaskFlow API, set a schedule without the classic start_date and catchup surprises, pass data between tasks, add retries, and test it with airflow dags test before deploying.
How Azure CNI Overlay, flat Azure CNI with a dedicated pod subnet and kubenet differ in IP use, routing and limits; how to size CIDRs for growth; and how to plan egress so clusters do not hit SNAT or IP exhaustion later.
Control where pods land on AKS so a zone or node loss does not take a service down. Topology spread constraints, dedicated node pools with taints, pod anti-affinity, priority and preemption, and how they interact with the cluster autoscaler.
Create a private bucket with sensible defaults, upload, download and sync folders, share a single file safely with a presigned URL, and turn on versioning so mistakes can be undone.
How Iceberg partition transforms work, why queries do not need partition columns, how to evolve a table from daily to hourly or add bucketing without rewriting history, and how to check pruning and compact old layouts when you want to.
Build an event-driven remediation loop. An Events rule catches a risky change, an OCI Function using resource principals fixes it and posts a notification. Includes IAM policies, a Python function, testing and the safety rails that stop automation fighting your engineers.
Split Azure Databricks Terraform into account, workspace and data layers, configure providers with Entra ID auth, create storage credentials, external locations, catalogs and grants, and avoid the state and permission traps that make plans destructive.
Choose between full clones, metadata clones, clones from backup and refreshable clones on Autonomous Database. Create and refresh them from the CLI, mask data for test, use refreshable clones for reporting offload, and know the limits that bite.
CPU-based HPA does not work for GPU inference. Scale vLLM or TGI replicas on waiting requests and KV cache usage with KEDA and Prometheus, pair it with the cluster autoscaler or Karpenter for GPU nodes, and hide multi-minute cold starts with warm capacity.
Install AWS CLI v2, sign in with IAM Identity Center instead of long-lived keys, use named profiles for several accounts, and always confirm the account and region before you run a change.
Replace dozens of per-project policies with a few attribute-based policies that compare principal tags with resource tags, protect the tags themselves, pass tags from IAM Identity Center, and test the result before rollout.
The IAM building blocks in plain terms, how a policy document is read, how roles are assumed, and a small worked example that gives an EC2 instance read access to one S3 bucket without keys.
Publish an internal service to other VPCs and accounts with PrivateLink instead of peering. Build the NLB-backed endpoint service, approve consumers, set up private DNS with domain verification, and debug the usual connection failures.
Install AzCopy, sign in with Entra ID instead of keys, copy and sync folders to and from Blob Storage, and read the job logs when a transfer fails part-way.
A first-day guide to the Azure CLI. Sign in safely, make sure you are in the right subscription, set a default resource group and location, and filter output with --query so you read only what matters.
Find out what costs money in a subscription, group spend by resource group and tag, create a budget with email alerts, and export daily cost data so surprises show up in days, not at month end.
Databricks spend shows as untagged in Azure cost analysis or chargeback reports. Find which tag path failed (pool, workspace, conflict, restart, serverless) and fix it.
A guided first hour in an Azure Databricks workspace. What each sidebar area is for, which compute to pick (serverless, all-purpose, job, SQL warehouse), creating a small cluster with auto-termination, and a first notebook that reads and writes a table.
Use data collection rule transformations to drop noisy rows, remove or hash sensitive columns, and route data to cheaper tables before it is stored, with a safe test-and-rollout process.
Use Azure Virtual Network Manager network groups and security admin rules to enforce organisation-wide allow/deny rules across many VNets, understand how they combine with NSGs, and deploy them without cutting off production.
What each VM power state means, why "Stopped" can still bill for compute, how to deallocate, start and resize from the CLI and PowerShell, and what changes (IPs, temp disk) when you do.
Use system.access.table_lineage and column_lineage to find every job, pipeline, dashboard and user that reads a table or column before you change it, and what lineage misses.
Use what-if output as a pipeline gate, manage resource lifecycles with deployment stacks so removed resources are actually deleted or detached, and protect stack-managed resources from portal edits with deny settings.
A step-by-step checklist for rolling a new NVIDIA driver across GPU node pools serving LLMs - compatibility checks, capacity headroom, GPU Operator upgrade policy, canary nodes, validation and rollback.
Deploy baseline resources (roles, Config rules, alarms, logging) to every account and region with service-managed StackSets, roll out with concurrency and failure tolerance, handle drift and failed instances, and avoid the classic StackSet traps.
Understand metrics, statistics and periods, create a CPU and a status-check alarm that emails you through SNS, test it on purpose, and avoid the beginner settings that make alarms noisy or silent.
What each common data file format is good and bad at, why Parquet is the default for analytics, where Avro and JSON still fit, and how to inspect and convert files with Python, Spark and DuckDB.
Turn a working notebook into a reliable scheduled job. Pick job compute, set a schedule and timezone, pass parameters, configure retries and timeouts, get alerts on failure, and run it as a service principal.
The notebook features you use every day. Language magics (%sql, %python, %sh, %pip), widgets for parameters, the most useful dbutils commands for files, secrets and notebooks, and how %run differs from dbutils.notebook.run.
Create or pick a SQL warehouse, write and save queries in the SQL editor, read query history and the query profile when something is slow, build a simple AI/BI dashboard and schedule it, with sizing tips that keep costs down.
Set up a dbt Core project, declare sources, build staging and mart models with ref(), add built-in tests, choose materializations, and generate documentation, with the commands you will run every day.
Enable change data feed on Delta tables, read inserts, updates and deletes by version or as a stream, apply them downstream with MERGE, handle retention and schema changes, and know when CDF is the wrong tool.
A first hands-on tour of Delta Lake. Create a Delta table, append and upsert data with MERGE, read the transaction history, travel back to an older version and restore it, with SQL and PySpark side by side.
No posts match these filters yet.
Showing the latest 36 of 1134 posts. Open a platform above or search to see them all.