|By Harald Zeitlhofer||
|July 29, 2014 09:45 AM EDT||
Do you have a PHP application running and have to deal with inconveniences like lack of scalability, complexity of debugging, and low performance? That's bad enough! But trust me: you are not alone.
I've been developing Spelix, a system for cave management, for more than 20 years. It originated from a single user DOS application, and has now emerged into a web application with hundreds of users, used nationwide as the official cave directory in Austria.
Just as many software projects: Spelix evolved from MS DOS to PHP-powered Web 2.0 with increased demand for functionality and scalability
Recently I applied for a job at Compuware. For a presentation during my job interview I prepared a case study about how to monitor and improve performance of Spelix with dynaTrace, a tool in Compuware APM's suite. I found more hotspots than expected, and it was much easier than expected to resolve them. I also killed two birds with one stone: Spelix is really performing now and I've got a cool, new job.
Let me share with you my experiences in that process and the best practices I've applied to bring spelix.at to its current stage.
The Challenge of Software Evolution
In Spelix I have identified six major scopes for performance optimization:
- Server Side Data Caching
- Client Side Data Caching
- Network Traffic
- Browser / CDN Cache
- Server Side Session Handling
To be more digestible, this blog is split into two. This post focuses on the first three performance improvement steps and the next one focuses on the last three.
Step #1: Optimize Database Access
In the early stages of an application, it may not be relevant how your database access is designed. As long as the amount of data is low, a poorly designed query or missing indexes may not really affect overall response times. Therefore, database performance is rarely a topic that comes up in many cases. Once it has become an issue, it may be rather complicated to be handled. It's important to place enough value in your database design right from the start. Here are some of my lessons learned and best practices:
Interaction of Views & Indexes
I don't want to get into too much detail about creating indexes as many other articles have covered this. But it's rather important to understand when an index is not used: be careful when using views in MySQL!
When should you use views? Views are perfect to create complex queries and store them for further use. Views are commonly used to prepare data for presentation, or even for data pre-selection based on user access rights.
When should you avoid views? While a WHERE clause on a simple view may cause an index to be used, this could fail in complex view, even though the WHERE clause is on the primary key for the primary table. Once your query gets too complex, MySQL creates a temporary table for the result of the view, and then applies the query on top of the view, without any indexes to be used. Be alert when your view contains commands like GROUP_BY, ORDER_BY, or UNION. So what? My key advice on this is: when you create a view, define possible WHERE clauses and check the execution plan in the database by using the EXPLAIN command. If your WHERE clause is on a column in a table marked as select type "primary", you are on the save side to use the view. If it's "derived" or "dependent subquery", your query might not use existing indexes. It could be better to execute the query code from the application. If you have executed the SQL statements directly from your business logic, create a data access layer that contains your query code. Consider executing multiple SQL statements and merge the data in your PHP code rather than using complex joins that may spoil your indexes.
Check and Eliminate Redundant Statements
Check if your SQL executions are really necessary! You might have executed your statement in an earlier stage, is there really a requirement to run it again? Would it make sense to keep the data in your current context instead of re-requesting it? The following screenshot shows the database statements executed by the PHP Application.
A very good metric is "Executions per calling Transaction" which makes it easy to highlight statements, which are called several times, maybe too often per transaction. If that number is greater than 1, you might want to dig deeper into your code and try to optimize that. In this example "select * from sys2" reads the settings for the current user, which is not going to change permanently. There is no requirement to run this query redundantly.
What to look for? Find your query invocations in your code and avoid repetitive executions.
Optimizations? Depending on the type of information, consider caching your data in your transaction, session storage or overall server side cache, as described in the next section.
Seeing the actual SQL Statements in the context of the request makes it easier to optimize executions of database queries.
For steps 2 and 3, and for further insight, click here for the full article.
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Oct. 10, 2015 07:00 AM EDT Reads: 5,933
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Oct. 10, 2015 06:00 AM EDT Reads: 815
The broad selection of hardware, the rapid evolution of operating systems and the time-to-market for mobile apps has been so rapid that new challenges for developers and engineers arise every day. Security, testing, hosting, and other metrics have to be considered through the process. In his session at Big Data Expo, Walter Maguire, Chief Field Technologist, HP Big Data Group, at Hewlett-Packard, will discuss the challenges faced by developers and a composite Big Data applications builder, focusing on how to help solve the problems that developers are continuously battling.
Oct. 10, 2015 04:00 AM EDT Reads: 520
Nowadays, a large number of sensors and devices are connected to the network. Leading-edge IoT technologies integrate various types of sensor data to create a new value for several business decision scenarios. The transparent cloud is a model of a new IoT emergence service platform. Many service providers store and access various types of sensor data in order to create and find out new business values by integrating such data.
Oct. 10, 2015 04:00 AM EDT Reads: 601
There are so many tools and techniques for data analytics that even for a data scientist the choices, possible systems, and even the types of data can be daunting. In his session at @ThingsExpo, Chris Harrold, Global CTO for Big Data Solutions for EMC Corporation, will show how to perform a simple, but meaningful analysis of social sentiment data using freely available tools that take only minutes to download and install. Participants will get the download information, scripts, and complete end-to-end walkthrough of the analysis from start to finish. Participants will also be given the pract...
Oct. 10, 2015 03:00 AM EDT Reads: 333
WebRTC: together these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at WebRTC Summit, Cary Bran, VP of Innovation and New Ventures at Plantronics and PLT Labs, will provide an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it may enable, complement or entirely transform.
Oct. 10, 2015 02:15 AM EDT Reads: 772
SYS-CON Events announced today that Dyn, the worldwide leader in Internet Performance, will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Dyn is a cloud-based Internet Performance company. Dyn helps companies monitor, control, and optimize online infrastructure for an exceptional end-user experience. Through a world-class network and unrivaled, objective intelligence into Internet conditions, Dyn ensures traffic gets delivered faster, safer, and more reliably than ever.
Oct. 10, 2015 02:00 AM EDT Reads: 661
WebRTC services have already permeated corporate communications in the form of videoconferencing solutions. However, WebRTC has the potential of going beyond and catalyzing a new class of services providing more than calls with capabilities such as mass-scale real-time media broadcasting, enriched and augmented video, person-to-machine and machine-to-machine communications. In his session at @ThingsExpo, Luis Lopez, CEO of Kurento, will introduce the technologies required for implementing these ideas and some early experiments performed in the Kurento open source software community in areas ...
Oct. 10, 2015 01:00 AM EDT Reads: 785
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome,” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Oct. 10, 2015 12:00 AM EDT Reads: 185
Today air travel is a minefield of delays, hassles and customer disappointment. Airlines struggle to revitalize the experience. GE and M2Mi will demonstrate practical examples of how IoT solutions are helping airlines bring back personalization, reduce trip time and improve reliability. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Dr. Sarah Cooper, M2Mi's VP Business Development and Engineering, will explore the IoT cloud-based platform technologies driving this change including privacy controls, data transparency and integration of real time context w...
Oct. 9, 2015 10:15 PM EDT Reads: 151
The IoT market is on track to hit $7.1 trillion in 2020. The reality is that only a handful of companies are ready for this massive demand. There are a lot of barriers, paint points, traps, and hidden roadblocks. How can we deal with these issues and challenges? The paradigm has changed. Old-style ad-hoc trial-and-error ways will certainly lead you to the dead end. What is mandatory is an overarching and adaptive approach to effectively handle the rapid changes and exponential growth.
Oct. 9, 2015 10:00 PM EDT Reads: 250
Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, will discuss the impact of technology on identity. Should we federate, or not? How should identity be secured? Who owns the identity? How is identity ...
Oct. 9, 2015 10:00 PM EDT Reads: 455
The buzz continues for cloud, data analytics and the Internet of Things (IoT) and their collective impact across all industries. But a new conversation is emerging - how do companies use industry disruption and technology enablers to lead in markets undergoing change, uncertainty and ambiguity? Organizations of all sizes need to evolve and transform, often under massive pressure, as industry lines blur and merge and traditional business models are assaulted and turned upside down. In this new data-driven world, marketplaces reign supreme while interoperability, APIs and applications deliver un...
Oct. 9, 2015 08:00 PM EDT Reads: 322
Electric power utilities face relentless pressure on their financial performance, and reducing distribution grid losses is one of the last untapped opportunities to meet their business goals. Combining IoT-enabled sensors and cloud-based data analytics, utilities now are able to find, quantify and reduce losses faster – and with a smaller IT footprint. Solutions exist using Internet-enabled sensors deployed temporarily at strategic locations within the distribution grid to measure actual line loads.
Oct. 9, 2015 06:30 PM EDT Reads: 148
The Internet of Everything is re-shaping technology trends–moving away from “request/response” architecture to an “always-on” Streaming Web where data is in constant motion and secure, reliable communication is an absolute necessity. As more and more THINGS go online, the challenges that developers will need to address will only increase exponentially. In his session at @ThingsExpo, Todd Greene, Founder & CEO of PubNub, will explore the current state of IoT connectivity and review key trends and technology requirements that will drive the Internet of Things from hype to reality.
Oct. 9, 2015 05:30 PM EDT Reads: 128
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data shows "less than 10 percent of IoT developers are making enough to support a reasonably sized team....
Oct. 9, 2015 04:00 PM EDT Reads: 254
You have your devices and your data, but what about the rest of your Internet of Things story? Two popular classes of technologies that nicely handle the Big Data analytics for Internet of Things are Apache Hadoop and NoSQL. Hadoop is designed for parallelizing analytical work across many servers and is ideal for the massive data volumes you create with IoT devices. NoSQL databases such as Apache HBase are ideal for storing and retrieving IoT data as “time series data.”
Oct. 9, 2015 03:45 PM EDT Reads: 512
Today’s connected world is moving from devices towards things, what this means is that by using increasingly low cost sensors embedded in devices we can create many new use cases. These span across use cases in cities, vehicles, home, offices, factories, retail environments, worksites, health, logistics, and health. These use cases rely on ubiquitous connectivity and generate massive amounts of data at scale. These technologies enable new business opportunities, ways to optimize and automate, along with new ways to engage with users.
Oct. 9, 2015 02:00 PM EDT Reads: 194
The IoT is upon us, but today’s databases, built on 30-year-old math, require multiple platforms to create a single solution. Data demands of the IoT require Big Data systems that can handle ingest, transactions and analytics concurrently adapting to varied situations as they occur, with speed at scale. In his session at @ThingsExpo, Chad Jones, chief strategy officer at Deep Information Sciences, will look differently at IoT data so enterprises can fully leverage their IoT potential. He’ll share tips on how to speed up business initiatives, harness Big Data and remain one step ahead by apply...
Oct. 9, 2015 01:45 PM EDT Reads: 568
There will be 20 billion IoT devices connected to the Internet soon. What if we could control these devices with our voice, mind, or gestures? What if we could teach these devices how to talk to each other? What if these devices could learn how to interact with us (and each other) to make our lives better? What if Jarvis was real? How can I gain these super powers? In his session at 17th Cloud Expo, Chris Matthieu, co-founder and CTO of Octoblu, will show you!
Oct. 9, 2015 01:15 PM EDT