Kye Hittle


Recent posts by Kye Hittle

5 min read

Data Lake Basics

By Kye Hittle on May 27, 2021 9:02:00 AM

Blogpost-Display image-May_Data Lake Basics

Terms and Themes

With Atlassian's upcoming release of Jira Data Lake for Jira Software Cloud, it's a good time to review the jargon we might stumble on in the reporting and business intelligence (BI) space. So let's jump into the (data) lake!

One word of caution: the BI industry has many players with varied opinions. Some terms get used and reused in multiple ways. One example is the emerging use of "lakehouse" - a combination of "data lake" and "data warehouse." Here we'll stick to as close to canonical as possible but expect to see terms used differently as you research.

Why does BI even matter? What are KPIs?

Your organization has systems (e.g. computer applications) which create and contain data. That data is extremely valuable for fact-based decision making in your organization. 

A CTO or CIO is able to more effectively allocate help desk head count with ready access to accurate metrics (also called Key Performance Indicators, or KPIs) like Mean Time To Acknowledge (MTTA) and Mean Time To Resolve (MTTR). (Note: MTTR is a tricky acronym. As Atlassian notes, there are at least four common incident management metrics that share this abbreviation! This stuff can be confusing...)

To provide these valuable, up-to-date KPIs to decision makers, we turn to BI. This industry is a dizzying array of technology components which take various approaches to achieving BI's primary objective: turning raw data into actionable insight. Often, we need to integrate multiple BI components to get from point A (data in the source system) to point B (reports used for decision making).

BI solutions often leverage a data lake or data warehouse to store business data.

What is a data lake?

A data lake is a central store of raw business data. The data lake is not typically used by the source systems whose data it contains.

The lake is designed to be accessed by tools like Tableau, PowerBI, and Qlik in order to analyze and produce insights from the data. We'll call these analysis and presentation applications "BI tools." To continue the lake analogy: if the BI tool is a fishing rod, then the data is the fish.

A data lake typically uses a file store technology but when it comes to Jira Data Lake, we don't really need to know much about the underlying tech because Atlassian Cloud takes care of choosing, configuring, hosting, and maintaining it for us. One less thing on our plate? Great!

All we need to do is connect our BI analysis and presentation tools (Tableau, PowerBI, Qlik, etc.) to Jira Data Lake. Boom! We're ready to start creating reports, graphs, dashboards, and whatever else we need to answer questions for our organization.

How is a lake different from data warehousing?

As mentioned earlier, some BI solutions use a data warehouse instead of a data lake. Some use both. While the line has blurred between the two, lakes are usually more unstructured than warehouses.

The initial data lake concept encouraged organizations to dump all of their raw data into the lake, including data from relational databases, flat files (e.g. CSV files), videos, and more. The promise that smart software and ever-increasing computing horsepower would eventually create solutions for accessing the overwhelming amount of data in the lake hasn't really come to fruition quickly enough. And many data lakes turned into data swamps. Lakes these days, like Jira Data Lake, are more purpose-built and have better designs for preventing a descent into swampland.

A data warehouse is more structured and normally designed with transformation processes on the front- and/or back-end that clean, normalize, and handle any other standardization before presenting it to our BI tools. These processes are represented by the "T" (Transform) in some more acronyms: ETL (Extract Transform Load) or ELT. The result is more predictable and accurate, but the cost and time to create these transformation processes is much higher.

Why use a data lake?

Why invest in this effort to centralize data in lakes or warehouses? Our BI tools can often connect directly to our application's database. Wouldn't it be easier to skip the lake/warehouse?

Eliminating the data lake or warehouse would simplify our solution design but experience has shown multiple issues with the direct-connect approach.

The most critical issue is often the potential load a BI tool can place on an application database. BI queries often require large swaths of data which can only be fulfilled through heavy workloads on the database. In addition, BI tools often don't optimize queries for performance. BI workloads can cause database contention and application stability should always be prioritized over BI needs. With today's easy-to-use BI tools accessible to a larger and less technical audience, this issue has only become more prevalent. Connecting our BI tools to a data lake prevents risking any application stability issues.

The next most common issue we see is needing to combine data from multiple systems. Since your organization doesn't just use one system, combining data across the organization is how so many powerful insights occur. For example, tying Jira KPIs to financial data is one way leaders can more easily understand technical metrics. But financial data is stored in the accounting system, not Jira. A direct connection to an application's databases only allows access to that system's data, preventing cross-system data analysis. While some BI tools allow you to perform "cross-database joins," performance is often unacceptable and some links are just not possible. Often the data from different systems needs to be cleaned and standardized before it can be linked for analysis. Doing this in a data lake/warehouse is far more efficient than attempting it "at runtime" in BI tools. When we first centralize our data we have the ability to combine data from as many systems as needed.

BI is all about trends over time. Some applications don't maintain much, if any, historical data. A direct connection to these systems doesn't allow for time-based analysis. The historical data simply doesn't exist. Lakes allow us to snapshot data at regular intervals in order to perform valuable time-based analysis.

Finally, with cloud apps like Jira Cloud, we don't have the option to connect directly to the application database. The only data access is often through APIs which can be slow for analysis and suffer from many of the same issues mentioned above. Jira Data Lake provides performant, safe data access.

Data lakes arose from the need for flexibility. No two organizations use the same systems or have the same data needs. Your organization's data needs will also change over time. The direct connection to an application database is too tightly coupled and doesn't provide enough agility to provide BI insights.

If you're wondering if this powerful new tool is a good fit for your organization, or have any questions about anything Atlassian, contact us, one of our experts would love to help!

Topics: blog management tips data business-intelligence data-lake jira-data-lake
6 min read

All in Good Time with Atlassian’s Team Calendars for Confluence

By Kye Hittle on May 17, 2021 11:23:52 AM

Blogpost-display-image_Team CalendarsAh, a fresh, new month. For so long there was always at least one day where my email inbox was flooded with many, many calendar invites for recurring company-wide meetings, holidays, and deadlines. After carefully clicking “Accept” on each invite, I’d think, “there’s got to be a better way.”

Atlassian’s Team Calendars for Confluence offers a great solution, and it's included with Cloud Premium subscriptions! Let’s take a look.

TEAM CALENDARS FOR CONFLUENCE

 

Image source: Atlassian

What is Team Calendars for Confluence?

The plugin adds a Calendars tab to each space and you can create multiple calendars using built-in or custom event types. Each user also gains a “My Calendars” page which rolls up all Team Calendars they’ve watched. This is centralized, always up-to-date, and customizable calendar management.

Why use Team Calendars?

Clear the clutter. While Team Calendars helps avoid periodically flooding everyone’s inboxes with invites, it also prevents tasking someone to reissue invites to new team members who onboard mid-year. Even those of us who aren’t new can avoid getting peppered with calendar updates when inevitable changes occur.

Visualize. Team Calendars display events as a live calendar, which is a visual metaphor instantly grokked by most everyone. Select between week, month, list, or Gantt-like timeline views. Assign different colors and icons to event types to further visually distinguish your layout. We often see clients using Confluence tables to list out dates. Tables capture the event data but require unnecessary mental overhead to comprehend and can’t be combined with other calendars to spot opportunities and conflicts.

Crowd-source your calendars. Team Calendars allow any user to add and edit events, keeping calendars comprehensive and accurate. Most calendar systems don’t allow this or it’s too cumbersome. In Confluence, it can also be restricted when needed.

Let’s TAke Control of Calendars

At Praecipio Consulting, we’ve helped organizations use Team Calendars for an incredibly diverse set of use cases. Here’s how we suggest you get started. 

Corporate holidays and time off (vacation, medical leave, volunteer time off, etc.) are often some of the first calendars created since they have major impacts across the organization. Keeping these events in context with your day-to-day planning in Confluence increases their visibility and prevents conflicts.

Holidays and time-off are just the tip of the organization-wide event iceberg. Take a look at your work calendar and you’ll see lunch & learns, committee meetings, submission deadlines (expense reports, timesheets, benefits enrollment, etc.), social events, and more. Centralizing all of this in Confluence can result in a major productivity boost and a calmer work life.

Next, each team should consider the events unique to their work and create logical calendars to match. Marketing teams need to keep content creation, campaign schedules, and ad runs coordinated. Dev and product teams always need to have their release schedule handy. Client-facing teams may need to schedule around their clients’ external schedule of milestones, holiday, and deadlines. IT and service desks will need to keep support professionals informed of planned maintenance and outages. Each team will find they have many calendars and events to keep track of – and they’ll likely do a better job when using Team Calendars versus the invite model imposed by most calendar systems.

PRO TIPS

  • Designate a single calendar as the official organization holiday calendar. Have all other teams add it to the Calendars tab in their spaces. It’s inefficient (and dangerous) to have many different “Acme Co Holiday” calendars! Remember, Team Calendars makes it easy to reuse calendars and combine the calendars into one view! Many organizations choose to have this calendar live in a Human Resources space.
  • If you use Jira to track time-off requests, you can setup Custom Event Types which display these requests from Jira on the calendar to avoid duplicate data entry!
  • Use the Custom Event Types which allow Team Calendars to display live sprints, releases, and more from Jira. Using JQL you can specify exactly what’s displayed on your calendar, automatically updating as Jira changes.
  • If you are working with a client and they can provide an .ics file (usually available as an export option from most calendar services), you can quickly import hundreds of events into a Team Calendar so you can keep tabs on their events.
  • If there’s an existing calendar system you cannot migrate to Team Calendars, you may still be able to display the calendar feed within a Team Calendar. See subscribing to third-party calendars. Examples include Outlook/Exchange, Google, Teamup, Opsgenie, and PagerDuty.

Using Your Calendars

Now that you’ve got calendars setup, you’ll always find them under the Calendars tab within your Confluence space. This tab rolls up all calendars in the space (including calendars linked from other spaces) so you can see holidays, time off, deadlines, and happy hours all in one place. 

But wait! There are additional convenient ways to access your calendars!

  • Embed a calendar into a Confluence page with the Team Calendars macro
  • Link to an existing calendar in another space so that it shows up in your space’s Calendars tab (example: most spaces will likely link to the official corporate holiday calendar)
  • Each Confluence user will see all of the calendars they’ve watched in their My Calendars page
  • Integrate Team Calendars into your personal calendar in Outlook, iPhone, etc. Share these instructions with your users!

MORE TIPS

  • Embed a calendar(s) into your weekly team meeting notes (automate this with a template). Many of our customers have reported dramatically decreased schedule conflicts when the calendar is right there, being reviewed regularly.
  • When viewing calendars in a space’s Calendars tab or all the calendars you’re watching in the My Calendars page, you can temporarily filter out unnecessary event types by unchecking the boxes displayed to the left of the type under its calendar. If you want to hide an entire calendar, click the menu (…) next to a calendar name and choose Hide Events.

Caution

Like all Atlassian tools, it’s easy and intuitive to get started with Team Calendars. Here are some more considerations to make it an even smoother journey.

Calendar Names. A Confluence space’s view permissions are used to determine calendar visibility by default. Team Calendars does not enforce unique calendar names. For admins and others who belong to many Confluence spaces, having 27 calendars all named “PTO” makes it hard to find the correct calendar. We recommend including the space name or key in each calendar name. For example, “PTO - IT Help Desk” and “PTO - Marketing.” 

Beware when deleting custom event types. Deleting a custom event deletes all events assigned that event type. Move events currently categorized under the event type to another event type before deleting.

Migration considerations. Atlassian does not officially support Team Calendars migration but you can export and import each calendar manually to move your calendars. Custom Event Types are great but if you’re migrating to a new environment, make sure you are using the latest version of Team Calendars in both environments, otherwise custom event types may be lost.

Help is here! There’s an entire section of documentation for Team Calendars. If you need Team Calendars licenses (or are looking to migrate to Cloud Premium, which includes Team Calendars), need to migrate your Confluence environment, or need assistance with any part of the Atlassian suite, get in touch with us!

Topics: atlassian blog confluence teams tips project-management confluence-cloud
2 min read

Why Digital Asset Management is Important

By Kye Hittle on May 14, 2021 1:37:00 PM

Blogpost-Display image-May_Why Digital Asset Management is ImportantWe're always looking for ways to keep track of our stuff, from old metal asset tags firmly glued to lids of the first "portable" computers to Apple's recent AirTag product release.

At work we call these "assets" because they cost money to acquire, maintain, replace, and are (hopefully) required for our organization's operation. (If assets are not being used, your digital asset management system should be highlighting that potential savings opportunity!) Keeping track of these items doesn't just make sense from a financial perspective, it's also required by law in many cases.

When it comes to asset management we're not just concerned with an item's current location. Surprisingly often, an asset's purchase price, age, vendor, warranty details, user assignment, support/maintenance contracts, service history, and any of hundreds of other details become critically important to keeping the asset—and therefore our business—running.

And we're not just talking about physical assets like desks, laptops, phones, tablets, tools, networking equipment, etc. The move to cloud means we can instantly deploy servers, licenses, and other IT infrastructure we'll never actually see or touch! How do I put an RFID tag on a cloud server?

With more devices and services being employed to operate our organizations every day, spreadsheets don't cut it. Given this amount of critical data to manage, the only way to keep up is to turn to digital transformation.

Traditional Configuration Management Databases (CMDBs)

The technology market has seen the introduction of many inflexible, expensive "solutions" to manage assets digitally. Traditional Configuration Management Databases (CMDBs) have failed to deliver the necessary transformative power:

  • IT is overpaying hundreds of millions of dollars in unused features in these legacy CMDB tools
  • Customization requires specialized consultants (quickly adapting to the changing needs of the business is a core tenant of digital transformation)
  • Legacy tools often result in slowing down the flow of work across teams instead of enhancing collaboration between them

Praecipio Consulting is transforming organizational service delivery with an Atlassian alternative built to deliver maximum value: Insight, now built into Jira Service Management. It is a modern, flexible digital asset management solution to easily define collaborative asset tracking that best fits your organization's needs, right in Jira.

Atlassian Service Management saves companies money by retiring their legacy tools. This explains why Atlassian is ranked as a strong performer in this market, having a strong strategy, and achieving a rapidly expanding market presence.

From employee and contractor onboarding to incident management to asset intelligence, Atlassian Insight for Jira Service Management can quickly get your digital asset tracking under control and flex to meet your constantly changing business.

Digital asset management done right doesn't just require the best-in-class solution, however. It's a cultural shift in the way IT is delivered as a service. Contact Praecipio Consulting to get started on your service delivery transformation now.

Topics: jira atlassian blog asset-management tips service-management insight digital-transformation jira-service-management
2 min read

Get early access to Atlassian Data Lake for Jira Software

By Kye Hittle on Apr 23, 2021 2:00:00 PM

Blogpost-display-image_Jira Data Lake Preview

What's a data lake?

Read up on the basics in our explainer.

At Praecipio Consulting we understand that the data contained within your Atlassian tools is a critical asset for your organization. To help customers more easily access their Jira data, Atlassian has developed Data Lake! As of March 2021, Data Lake is available to preview in Jira Software Cloud Premium and Enterprise.

Warning! Beta software should not be used for production purposes. Breaking changes are likely as Atlassian tweaks this functionality based on user feedback. Not all Jira data is currently available and permission levels are limited but Atlassian is quickly working through its roadmap. In addition only English field names are available, as of now. Therefore, any information presented here is subject to change.

Data Lake allows you to quickly connect the best-in-class business intelligence (BI) tools you've already invested in to query the lake directly.

Compatible BI Tools include:

  • Tableau
  • PowerBI
  • Qlik
  • Tibco Spotfire
  • SQL Workbench
  • Mulesoft
  • Databricks
  • DbVisualizer

Jira-Data-Lake-preview

Data Lake uses the JDBC standard supported by many BI vendors. Supporting an open standard provides tremendous flexibility and power in reporting on your Jira projects.

Once you've identified the components of your BI solution, you'll follow three basic setup steps:

  1. Configure the JDBC driver
  2. Connect your BI tool(s)
  3. Navigate the Jira data model

You'll need your org_id and an API token for your Jira Cloud instance. Except for creating an API token (if you haven't already), there's no config required within your Jira instance. There are instructions for connecting to various BI tools in the Atlassian community Data Lake Early Access group. In addition, you'll find posts and diagrams to assist in answering business questions using Jira's data model.

If you're a Premier or Enterprise customer and would like to access the Early Access Program for Data Lake, complete this form to request access. You can also post questions and feedback for the devs in this group.

Are you interested in unlocking the power of data stored in your Atlassian tools? We're a Platinum Atlassian partner with years of experience helping customers leverage their Atlassian investment for even more value, so get in touch!

Topics: jira atlassian blog enterprise jira-software atlassian-products business-intelligence data-lake

Praecipio Consulting is an Atlassian Platinum Partner

This means that we have the most experience working with Atlassian tools and have insight into new products, features, and beta testing. Through our profound knowledge of Atlassian environments and their intricacies, we can guide your organization as you navigate these important changes.

atlassian-platinum-solution-partner-enterprise

In need of professional assistance?

WE'VE GOT YOUR BACK

Contact Us