2016-08-24---Planning-your-ETL-in-Qlik

Planning Your ETL in Qlik

Qlik has a powerful script engine which allows you to bring information in from a variety of sources. It’s included within Qlik and because you load all the data into memory, once saved it is effectively a portable business intelligence (BI) solution. Of course in the real world the file would remain on a server where all the decision makers can access the information held within.

Naturally, one would assume each dashboard would bring in the data needed from its sources and this can be the case. However, when you’re planning a larger scale strategic implementation this becomes less efficient.

Qlik has a simple yet effective option to allow you to develop these large suites of BI dashboards. As mentioned, Qlik stores the information into memory and also has the ability to export data tables in a special format which is already optimised in the standard Qlik file format (QVD). This optimisation reduces the data size without the need for compression. A Qlik dashboard file can now be seen as something different; it becomes a script which moves a set of data from one location to another and perhaps transforming it as well.

For large deployments we would recommend creating three layers (three Qlik dashboard files):

  1. Extract
  2. Transform
  3. Dashboard Load

The Extract layer would take information from the source systems and place it straight into a QVD file. Here, the latest information from the source system can be added to the data retrieved previously – also known as incremental loading. This method is far more efficient than taking the whole history each time you re-load. It’s important at the extract stage that no data is changed, combined or additional fields added. This way you can easily test that your saved data matches the source file.

The next step is Transformation.  Here we read the previously saved data and get it ready for consumption for a dashboard. As before we would do this incrementally for large tables that are added to over time. Transformation would include; joining tables together, adding flags and calculations.

Finally the Dashboard Loads the transformed data it requires. As the data is fully prepared in the transformation stage, it can be loaded as ‘optimised’ which means it happens very quickly.

When storing QVDs in the Extract and Transform layers it’s good practice to split large tables into manageable chunks. These tables typically contain your transactional data which grows over time (commonly referred to as FACT tables). These bite sized chucks depend on the data volume although we would typically recommend monthly files – this makes archiving data in the future easier. Also, if your dashboard only requires information from the last twelve months, you don’t need to load a QVD file which potentially could contain many years of data, which of course would slow down your process.

The benefits of this type of data load structure include:

  • Your source systems are only accessed once, reducing the demand on their resources
  • You can validate the accuracy of each stage of the process
  • QVD files are built up over time and are optimised
  • Several dashboards can re-use the same data which is both efficient and consistent

If you have any questions or you’d like more information on how Qlik business intelligence can help your organisation, please don’t hesitate to contact us on +44 (0)330 363 9900 today.

Thanks for reading!

Richard Pearce, Senior Consultant.

2016-08-03 - What makes a great Qlik dashboard

What makes a great Qlik Dashboard

What makes one Qlik Dashboard stand out from the rest?

It’s probably best to start at the beginning and ask the question: what is a dashboard? There are lots of ideas out there: “a single view, like the dashboard of your car, giving you the important facts you need in a clear, concise way” is probably the most popular definition. When it comes to business however, a single page dashboard often asks more questions than it answers. A simple set of dials doesn’t give you the full story, just the ending. So perhaps a single view is just one element of the bigger picture. This would take us neatly onto the “Dial to Detail” style of reporting, suggesting a top down approach to analysis. If the “dial” gives you the ending then it would make sense that the “detail” is the story, but is that the best approach to managing performance?

Qlik Dashboard Ometis 2

Effective Performance Management

Is performance management about investigating why something has happened (or possibly gone wrong)? Or should the focus be more on prevention and a continuous journey towards more efficient operational processes and increasing your organisation’s skills? Having the relevant data and being able to supply this information to the right person to analyse is key to success, although I’m always curious when organisations employ a team of analysts. In my experience these teams usually began life as report creators who churned out information by request. These teams are often asked to shift focus and spend more time analysing the information, they are analysts after all. The reason I find this shift somewhat curious is because I’m not sure how exactly that’s supposed to work. I believe that it is those who are managing the front line people and processes who are best placed to complete the analytical task and make effective changes based on their findings. I believe a front line manager knows their own area and information should act as an additional tool in this process.

Qlik Dashboard Ometis 3

Making It Personal

Mobile phone apps have been popular for a while but why is that? One reason is because apps allow every person to customise their phone to their own needs. Not too long ago when you bought the latest phone you were stuck with the manufacturer’s own calendar, calculator, task list etc. Now that’s changed with the advent of smart phones and what seems like a limitless supply of apps. My phone has become very personal to me. If I were to swap my phone with someone else’s, even though the phone may be the same make and model, the apps I inherit will probably be of little use to me in my daily life because they don’t meet my needs.

This makes me think that reporting performance information has to be personal to succeed. For example, let’s say we have three people as our internal information customers; the CEO, Sales Director and Operations Director. We could safely assume the Sales Director and Operations Director will want very different information in their regular reporting so two reports are created, one for each, and both reports also go to the CEO who has to then take the bits of information from both reports which interests them. People are asked to gather information far too often. Organisations I’ve been involved in have anywhere up to 100 regular reports being produced and few are aimed at an individual or role. Most cater to a service or department.

Qlik Sense Dashboard Ometis

Have you ever been on the internet looking for something slightly out of the norm? You cast your hook, see if anything bites then you reel back in and try again. That process can be very wearisome and it doesn’t take too long before you feel frustrated that the information you need isn’t at your fingertips. Information gathering can be very frustrating for an individual if they’re having to, in effect, create their own dashboard by collecting information from a variety of sources.

The takes me to what I believe is the answer. What makes a dashboard useful, usable and used is that it’s designed around the person or role, providing only required information in a format that is effective The fewer roles each dashboard is targeting at the better. with our Qlik Dashboards you can do just that. For more information on how we can help your business achieve effective data analysis, get in touch today.

What do you think makes an effective dashboard? Tweet us @ometis_ltd.

Richard Pearce, Senior Consultant

2016-05-04---Qlik-Sense-Tour----Top-5-Reasons-to-come-to-the-Qlik-Sense-Tour-2016-V1-3

Qlik Sense Tour 2016 Highlights

We’ve had a few weeks to let the dust settle after our amazing Ometis Qlik Sense Tour took place at Hotel Football on the 29th June 2016. We had a great day and an even better turn out. Everyone had a real interest in what Qlik and Ometis had to offer them, and their respective businesses and the day was a huge success.

A couple of highlights of our day are detailed below, just in case you missed out!

Keynote speakers

Ometis’ own Andy Patrick – Head of Sales and Qlik’s David Sedgwick – Principle Solution Architect gave the keynote speeches and demos explaining how by connecting, people, data and ideas you can uncover new data driven possibilities and see the whole story that lives within your data.

Customer Presentation

Gary Higgins- Financial Director at Pipers Crisps explored how his company are using Qlik to deliver detailed analytics across their entire organisation, towards their ambitious goals over the coming years.

 

Hands-on workshop

The hands-on workshop led by Chris Fitzpatrick allowed our delegates to experience Qlik Sense in action. Everyone got to experience Qlik’s platform for themselves, by putting in their own data. We put the power of data discovery right into their hands and helped them uncover new data driven possibilities.

With step-by-step instruction, guided by our experts, they were able to:

  • Build a fully functional dashboard application
  • Easily navigate their own application
  • Use interactive storytelling to present their findings

 

Networking Opportunities

As always these events are a great way to network with peers, Qlik, Ometis & and current Qlik customers who share similar day-to-day challenges. All questions were answered by the experts and everyone found out more about the wonderful world of Qlik Sense.

Overall, a great day and we can’t wait to see you at next year’s event!

2016-07-15-blog-Qlik-Sense

QS 3.0 New Features Blog

The new release of Qlik Sense, version 3.0, was released at the end of June and makes it even easier to explore your data and provide insight to your organisation.

Read below for our summary of the new features included in the latest version of Qlik Sense.

Visual Data Preparation

With Qlik Sense 3.0, self-service data exploration goes to the next level with visual data preparation. Previously, to link tables of data together took a bit of know-how. Now the tables are shown in their own “bubble”, meaning that you can drag tables that you know should be linked together and Qlik Sense will associate them based on the data.

1

Figure 1Table “bubbles” linked together in the new Visual Data Preparation window

Now you can leave the complicated job of modelling your data to Qlik Sense and concentrate on what is important, analysing your data.

Branding

Add some pizazz to your Qlik Sense apps by branding them. Add your company logo and colours to your Qlik Sense apps making your corporate applications look more professional.

2

Figure 2Apply branding to your Qlik Sense Apps

Visual Search

Search has become visual in Qlik Sense 3.0. With the addition of charts to the standard search tool, you can now save time searching through sheets and sheets to find the one you want. You can even search for chart types and Qlik Sense will return the most relevant results!

3

Time Aware Charts

Charts in Qlik Sense have always helped the user build something powerful in just a few clicks, but Time-aware charts in Qlik Sense 3.0 take this a step further.  You no longer need to build your own calendar tables and pre-select the most appropriate granularity of date to use in your chart.

With Time-aware charts you simply add the original Date field to the chart and Qlik Sense will automatically work out the best way to display it.  It could be days, weeks, months or years – Qlik Sense will automatically scale the chart for you.

In addition to this, it will also understand if you have missing data points where you may want to show a continuous scale and auto-fill the gaps. Line charts are available in this release, but you can expect to see other chart types with this functionality in future versions.

No selections:

4

2015 & 2016 selected

 5

Widgets

This new feature gives us, the developers, the freedom to create what the customer wants but doesn’t yet exist in Qlik Sense. Widgets can adapt to the shape and size and the customisation possibilities are endless. In this example we’ve made a KPI where the icon can relocate itself or disappear if necessary depending on the size and shape of the widget. The icon and the colours can be changed and the second measure is optional, and this is only the beginning!

6

The widget editor allows us to use HTML, CSS and Angular.js to create and edit widgets.  It’s a comfortable place for a developer and gives us all the tools we need to preview our widgets as well as to connect them to a published app to test it with real values.

This is just tip of the Widget iceberg – the possibilities are endless!

 7

 

DataMarket

Qlik DataMarket is a revolutionary new Data-As-A-Service offering from Qlik.  It allows direct access to a range of standardised information that can help supplement your analytics.  With Qlik Sense 3.0, Qlik have included access to four new premium data packages:

  • Historical stock prices from major stock exchanges
  • Financial data from companies worldwide
  • Population indicators for India’s states and districts
  • Population of Canada by provinces or territories

While we expect Qlik to continue to add in population and demographic data of new countries fairly regularly, it’s great to see some additional company and financial markets data being released.

We can see some great use-cases for this.  Imagine supplementing your ERP or CRM data with the latest stats about your Customers or Vendors.  For some of our Financial Services customers, having the ability to get single-source access to financial markets data will make a big difference in how responsive they can be when making data-driven decisions based on Qlik dashboards.

We’re looking forward to seeing what other new packages are made available in the coming months.

9

 

2016-06-15 - Choosing the right chart

Choosing the right chart

Choosing the right chart to best analyse your data and showcase your results can be tricky. Whilst data visualisation is incredibly useful, faced with the vast array of options it can be an intimidating task.

The best place to start is selecting the right type of visualisation. For this, you can choose from the four main types: comparison, composition, distribution and relationship. Then you can decide on the chart that best displays your results. At Ometis, we can even help you customise your dashboard to reflect your company’s branding.

 

The Four Main Types of Visualisations:

Comparison
Comparison charts are used to compare the magnitude of values to each other and can be used easily to find the lowest and highest values in the data. It can also be used to compare current versus old values to see if the values are increasing or decreasing. Common questions are “what products sell best” and “how are our sales compared to last year”.

Charts include:

  • Variable width chart
  • Bar chart – horizontal or vertical
  • Table or tables with embedded charts
  • Line chart
  • Circular area chart

 

 Composition
Composition charts are used to see how a part compares to the whole and how a total value can be divided into shares. Composition charts shows the relative value but some charts can also be used to show the absolute difference. The difference is between looking at percentage of total and value of total. Common questions are “how big of a market share do we have in a region” or “what areas are our budget divided into”.

Charts include:

  • Stacked bar chart
  • Stacked area chart
  • Pie chart
  • Waterfall chart
  • Tree map

 

Distribution
Distribution charts are used to see how quantitative values are distributed along an axis from lowest to highest. Looking at the shape of the data a user can identify characteristics such as the range of values, central tendency, shape and outliers. It can be used to answer questions such as “number of customers per age group” or “how many days late are our payments”

Charts include:

  • Bar histogram
  • Line histogram
  • Scatter plot

 

Relationship
Relationship charts are used to see the relationship between the data and can be used to find correlations, outliers and clusters of data. Common questions are “is there a correlation between advertising spend and sales of our products” or “how do expenses and income vary per region, and what’s the deviation”.

Charts include:

  • Scatter plot
  • Scatter plot bubble size
2016-06-03-Qonnections

What did we learn at Qonnections 2016?

Three days, hundreds of seminars and break-out sessions, and thousands of attendees. Qonnections is the yearly global partner and customer event hosted by Qlik. You can expect to see insight from the most important people at Qlik and in the Data Visualisation world.

A big part of what we learn from the leaders at Qlik is the direction they are going in. The bulk of this blog will be focused on the main topic of Qonnections 2016 – the improvements to the Qlik Analytics Platform coming soon.

 

Do you want to do more with your own data?

Coming in June 2016 is the latest update to Qlik Sense. Version 3.0 brings even more improvement to how easy it is to bring your own data into Qlik Sense.

You can now do more when combining different data sources without having to write a single line of script. Qlik Sense will identify how likely data is to be related to each other, creating the relationships between the data without you having to. Qlik Sense will also identify date fields and create calendars for you to use, and if you put these into a line chart you can now zoom in and out to look at particular time periods.

Along with your own data you also have access to cleansed data through the Qlik DataMarket. Coming soon is more detailed financial markets information covering 35 major stock indices, giving you financial information on the associated companies. More data on other topics will be available later in the year regarding Healthcare and Retail for you to make use of in your own analysis in both QlikView and Qlik Sense.

With Qlik Sense version 3.0 search gets smarter! Searching in a Qlik Sense app will not just result in field names and values but also now charts where they are being used. You no longer need to remember in which chart you found a specific value, just search and Qlik Sense finds it for you!

 

Like to get your hands dirty with code?

If you are a developer, then updates to the Qlik Analytics Platform also brings additions to the Dev Hub. Widgets are a new way of creating simple visualisation extensions without having to know any javascript; only HTML and CSS is used here – helping you modify the look and feel of your charts with little effort.

Developers are also getting improvements to how they interact with the back-end APIs. A new javascript library, Enigma.js, is coming too which will make interacting with the Engine API easier and more straightforward. Other APIs will take advantage of this in later versions.

Keeping up with the style standards at Qlik becomes easier with the release of Leonardo UI (javascript and css library) containing all

of the standard Qlik objects (e.g. buttons and switches) meaning that using these styles you can make your visualisations look and feel exactly like standard Qlik Sense, and have these objects updated as Qlik bring in updated styles.

 

Want to be able to access your data wherever you are?

Qlik is keen to emphasise that they are taking Qlik Cloud very seriously and are approaching Qlik Sense with a “cloud first” mentality.

Qlik Cloud is being improved continuously, with Qlik Cloud Business coming in the 2nd half of this year and Qlik Cloud Enterprise planned for 2017. Streams, which can be used to separate your apps into sections within Qlik Sense Enterprise, are also coming to the cloud. This will also mean that you will be able to control who is able to access your apps, as opposed to sharing all apps with all authorised cloud users.

Expect to see many more improvements in Qlik Cloud in the future, giving you the maximum flexibility on where and when you want to analyse your data.

 

To wrap up…

There is plenty to look forward to in June with Qlik Sense version 3.0 and beyond. It has never been easier to analyse your data, whether on your work PC analysing database information or in the cloud using your iPad doing some ad-hoc analysis of your spreadsheets.

Developers and data visualisation experts have plenty to dig their teeth into, with more ways of extending the Qlik Platform using web development skills they know already.

Whatever stage you are on your data analysis journey, wherever you are, Qlik has got you covered.

blog (2)

Top 5 Reasons to come to the Qlik Sense Tour 2016

Join the Ometis team at the Qlik Sense Tour in Manchester this June, and give your business the tools to create visual analytics that will shape your decision making for the better. Here are five more reasons to come along:

1. Learn about the benefits of Qlik Sense

You’ll get the chance to learn about Qlik; an analytics platform that allows you to use internal and external data to create, build and present invaluable visualisations for your company. This versatile platform collates data from a vast array of sources and uses it to create intuitive dashboards, making it easier for you to visualise all areas of your business.

2. Hear from our inspiring speakers

There’ll be a host of speakers from Qlik and Ometis to learn from, as well as our own customer presentation that will be provided by Gary Higgins, Financial Director at Pipers Crisps. Find out how Pipers Crisps are using The Qlik Platform to deliver detailed analytics across their entire organisation and towards their goals of having a stockist in every postcode throughout the UK.

Each expert will provide new insights into use of The Qlik Platform, providing current examples and case studies. Feel free to network with the speakers and have all your questions about Qlik answered on the day.

3. Hands on workshop

Want to see how it all works? There’ll be a workshop in the afternoon session, giving you the chance to see first-hand how Qlik can help your business. Demos of Qlik Sense will be available to watch on-screen, but trialing the technology yourself will give you a true taste of what it has to offer. The simple interface is easy to follow and you can add as much or as little data as you like; from info on sales figures and targets to expenses and customer trends, or any business data that you need to analyse. Visualisations are created instantly and you can modify and adapt them as and when you like with more data, ideas and figures. You’ll get the chance to:

  • Build a fully functional dashboard application
  • Easily navigate your application
  • Use interactive storytelling to present your findings

If you want to come prepared or can’t make the specific date then please feel free to download the desktop edition here, and contact Ometis for any support or guidance needed.

4. Networking opportunities

You’ll get the chance to network with Qlik, Ometis and your peers. Meet other like-minded visitors from your industry and share in their knowledge, learn from their expertise and pick up new tricks and techniques. New opportunities can often arise from networking as you make more connections and raise your business profile.

5. Free sign up!

It won’t cost you a penny to register! Enter a few basic details to sign up to the event and we’ll see you at Hotel Football in Manchester on 29th June at 9am.

To find out more and to register – click here.