|By Mark R. Hinkle||
|August 31, 2004 12:00 AM EDT||
It's 2:00 a.m., you're working on that critical presentation, and the power goes out. Since you moved your power supply to your significant other's computer, you just lost all your work.
We've all been there one time or another. Then the real trouble starts: not only haven't you saved your work in an hour but lo and behold your PC won't boot back into your operating system. As the cold sweat drips from your brow you realize that in addition to losing your presentation, you've also lost your financial records, calendar, and more data than you could ever hope to replace. Before it happens to you again you need to have a desktop backup strategy.
This scenario resonates with many, if not all of us. It's not an individual problem; it's a computer-user problem common among suits and "propeller-heads" alike. That's why I'm focusing on ways to avoid desktop disasters this month.
In addition to focusing on backup disasters I'm going to delve a little more deeply into the command line to solve some of these problems. I don't want to intimidate anyone who's more comfortable in the GUI world, but there are many easy-to-use and powerful command-line utilities that are at your disposal in most Linux distributions. Also, I have been recently inspired by Doc Searls (http://doc.weblogs.com) and his movement for DIY (do-it-yourself) IT; check out his IT Garage (http://garage.docsearls.com/). This idea isn't fascinating or interesting to me because it's new; it's because I come from the do-it-yourself generation, the one that makes it possible for a DIY channel on cable and 101 home improvement shows to exist. What's interesting and noteworthy is that, with a little knowledge, you can save yourself a lot of money by doing some of these tasks yourself. So with a nod to my fellow do-it-yourselfers, let's explore the world of Linux desktop backup strategies.
Desktop Backup StrategiesThere's an old saying, "An ounce of prevention is worth a pound of cure." This phrase could just as easily have been attributed to a system administrator after a server crash if it wasn't first attributed to Ben Franklin. More often than not we as computer users have not taken the most basic precautions with our personal computing environments. The bottom line is that there is no foolproof way to keep our PCs from crashing, so backups are critical. Please follow me as we take a walk down backup lane.
DataThe first step in developing a backup strategy for your data is to figure out where your data lives on your system. By default almost all your user data is stored in your /home/$USER directory ($USER just means the user you log in to Linux with). Once you confirm that this is where your information is, you can decide how you will make backups. In my case, I have a DVD/RW on my laptop. I try to keep the bulk of my data in one directory but it's actually a fat32 partition that I access from Linux or Windows. That directory is /windows/D/. I simply launch my favorite CD burner software k3b (www.k3b.org) and create a data project to copy all my critical files to a DVD or CD. Its drag-and-drop interface allows me to make copies easily, but that's only one of the many ways to archive your data. You can also copy to a second hard drive or other storage device or even back up your data over the network. However, the method is not nearly as important as remembering to do it, or at least scheduling the system to do it on a regular basis (see the sidebar How to Automate Backups).
Operating System BackupsMost of you who have read my previous columns will know I'm a Linux LiveCD junkie (www.linuxworld.com/story/45259.htm); Linux LiveCDs can be very valuable when backing up your operating system. One method for creating backups is to use the handy dandy Knoppix CD (www.knoppix.org) and then do a complete disk copy to an extra hard drive (see Figure 1). Now this is not the fastest way to do things, but it catches everything on my hard drive (including all my data) and it's very easy to verify the contents of the backup. In this scenario I'm going to use the example of a PC with two hard drives. The first hard drive will be where your operating system resides and the second hard drive is where we will store your data. Not everyone will have a second hard drive but in these days of cheap storage it's not cost prohibitive to have a second internal or an external hard drive. For a 60GB hard drive there are many options well under $100. Since you don't have to invest in backup software with these techniques, you'll most likely have the extra cash.
How to Back Up Your Hard Drive Using KnoppixYou can create backups of your hard drive in a variety of ways; however, copying the entire contents of one local drive to another can create the most thorough backup. To start, boot your computer using your Knoppix CD. You should see both hard drives on your desktop and if you open your shell you can sudo su to gain root (super-user) access. You can also discern which drives are which by their contents, which should be browsable using the Konqueror file manager. Make sure that once you have discerned which hard drive is which by browsing them that you unmount them so you can copy one drive to another. One drive is likely to be /dev/hda and the other is probably /dev/hdb. You may need to research them a little more thoroughly to be sure. To copy one drive to another simply type the following command:
dd if=/dev/hda of=/dev/hdb
Imaging Your Drive from Another ServerYou may not have room for a hard-drive image on your local machine and may choose to keep it on a server so that you can migrate it to newer hardware or use it for other PCs in your enterprise. Once again you can use the same dd command in conjunction with a variety of remote access protocols to reimage your disk. While I have done this a few times, I defer to an expert on this topic, J.H. Moore, who has put together an excellent how-to at www.okmoore.com/imagedrive.html, mirrored at knoppix.net/docs/index.php/ImageYourHardDriveUsingKnoppix.
Give Your Hard Drive a KickStartInevitably you'll have to reinstall your operating system on your desktop Linux PC; in a larger environment you may have to do it for many machines. Part of a good disaster plan is a fast recovery, that's why you may want to use Red Hat's KickStart to automate the reinstall or even the initial reinstall of systems. The process is to build one PC that has your preferred configuration (this is called the build machine), then use that template to "kick off" additional installs. This automates the installs and minimizes human interference (and consequent mistakes). This is a good measure for fixing failed machines because you basically maintain one master machine and then let it configure your additional PCs painlessly. This tool was designed for use with Red Hat Linux, but many people have hacked it for other distributions. A search on Google for "Kickstart Linux" will yield a bounty of information on the subject. For more information, check out the Kickstart (listman.redhat.com/archives/kickstart-list/) mailing list as it's a good place to ask questions and get ideas on what other Linux users are doing with this tool.
Other Tips and TricksBackups are obviously essential to recover from a desktop disaster, but here are some tips to prevent you from having to use your backups or fix your installation before you resort to a restore of your system.
Protecting Files: Making Data Read OnlySometimes we have data that we store but never alter. In those cases it could be advantageous to make the data read only so it doesn't get overwritten or deleted. You can make your data read only by using Linux permissions. Every file and device on the system is controlled by permissions, and before a file is executed, written, or read there it checks to make sure that that action is allowed.
Users and GroupsEvery process on a Linux system is executed by a user from the super-user root to a user with restricted permissions, so he or she can't compromise other parts of the operating system; in my case, this user is mrhinkle. Each of these users is part of one or more groups, which makes it easier to share resources and still enforce permissions. All information about users and groups is kept in /etc/passwd and /etc/group/. Now if you do have data and you don't write it often, or maybe it's just an archived copy of a presentation, you could change the permissions for the files to be "read only." That way it's harder, but not impossible, to be overwritten or deleted. Do this by using the chmod command. Chmod sets permissions for a file. The syntax is typically chmod ### file where ### are the permissions for owner, group, and all other users. For fields that I don't want to get overwritten I occasionally set that number to 444, where the four indicates read only.
In the example below I've created the file example.txt using the vi editor. Once I created the file I used chmod to set the files to read only so that I don't overwrite the data there. Notice that when I try to remove the file I also get warned. This isn't a foolproof method and I could have answered "y" to the rm dialog and the file would be deleted, but the warning should help me to think twice before I edit the file.
mrhinkle@linux:~> vi example.txt
mrhinkle@linux:~> chmod 444 example.txt
mrhinkle@linux:~> rm example.txt
rm: remove write-protected regular file `example.txt'? n
You may also want to be extra cautious and have the files owned by another user with just read-only attributes. For example, if I normally work as mrhinkle, I could have the files owned by root. That way, the only way I could delete them would be if I was logged in as root. You may even want to create an archive user, then use the shown command to "change owner" to archive. Keep in mind that to do this you must have permissions for that file so it's easily done as the root user. The syntax for this would be:
root@linux:~> chown archive example.txt
Keep in mind that I first created the user archive, then changed the ownership of my files.
Creating a Boot FloppyMost distributions offer various utilities for creating a boot floppy in case a misapplied kernel update or other disaster strikes. If you want to find out how to make your own, you can try this method. It takes your kernel image and copies it to a floppy disk.
Step 1: Find the KernelFor the most part, your kernel is usually going to be in /vmlinuz or /boot/vmlinuz (on my SuSE 9.1 installation it's /boot/vmlinuz). This is a soft link to my kernel, which is vmlinuz-2.6.5-7.95-smp.
Step 2: Copy the Kernel to a FloppyYou can do this by copying the kernel image to your floppy; in most cases this will be /dev/fd0.
dd if=/vmlinuz of=/dev/fd0
Step 3: Set the kernel image on the floppy to the location of your root system.rdev /dev/fd0 /dev/hda7
Your root filesystem may be some-where other than "/dev/hda7". You might find that your Linux installation is somewhere other than "/dev/hda7". I found out where my Linux installation was by changing to the root user and using the fdisk utility tool, and listed my partition table.
Simple Backup ScriptMost of us are used to our point-and-click environments, but sometimes it's easier to use the command line. However, if you're like me, you forget the syntax to the commands and make mistakes that cause the commands to become troublesome. That's why I took a little time to find a simple script to help you create a compressed archive directory complete with date. It also requires that you write a shell script. I chose to write the following script I called "arcive":
Step 1: In your favorite text editor type the following three lines:
tar czvf $1.$(date +%Y%m%d).tgz $1
Step 2: Save the file and make sure it's executable. I saved my file and called it arcive. (Since I have some other files called archive I decided to change the spelling so I wouldn't be confused. You could call it whatever you want.) I had to change the file to make it executable by using the chmod command:
chmod 755 arcive
Step 3: Now the easy part: I'm going to make a backup of my Firefox Web browser folder so I can preserve all my bookmarks and plug-ins as well as copy them to another test PC. The format for doing this is to simply enter:
where directory name is the name of the directory you want to archive. Since I archived a directory called Firefox I got the following result:
Notice that the name has the date (July 12, 2004) in it so I can easily track when I made the backup. This is a simple script that I find very useful as a noncommand-line guy; it makes it easy to archive directories.
How to Automate BackupsThere are as many ways to back up your data as there are types of data. Here's a quick way to back up your data that's mirrored in the exact same format that it's stored in your home directory. Keep in mind, this is from a command-line interface. You'll be using two commands: rsynch and crontab. They're probably installed on your system already, but if not you may want to use Google to search for more information on their installation and use.
Rsynch is used to synchronize with files at another location. In this simple example, we'll be synching to another directory on the hard drive. In this example I will be synching my data from my /home/mrhinkle/data/ directory to a drive I have mounted at /mnt/backups/.
Cron is a daemon that executes scheduled commands. In this case, we're using it to schedule backups, but you could use it to schedule the upload of files to a Web site, to archive mail, or for a variety of other tasks.
To start this exercise, type the following at a shell prompt:
then add a line in the format below. Keep in mind that the anatomy of a cron file looks like this:
30 4 * * * rsync -a /home/mrhinkle/data/ /mnt/backups/
30 4 * * *
This is the line that indicates the time at which to run the commands further down the line. Remember that your PC needs to be turned on at this time for it to run. (This should be evident but sometimes we forget the basics when we enter new territory.) The first number is the minutes; the second the hour; the next three asterisks indicate day of the month, month of the year, and then the days of the week (acceptable values here are 0-6 with zero equal to Sunday through Saturday. I used the wildcard "*" to indicate every day of the month, month of the year, and day of the week).
This is the command. Run rsynch, which will copy the data from one directory to another and only synchronize the data that has changed since the last synch. In the case of this example I put all my data in one directory. Other things you may want to back up are bookmarks and browser settings. (I use Firefox so all my browser preferences are saved in /home/mrhinkle/.mozilla/.)
This is the data I want to synchronize. In this case the data is stored in /home/mrhinkle/data/.
This is the location I want to synchronize to. In a desktop PC this might be a second hard drive or ideally a remote file server to ensure additional redundancy. I have many different configurations so this is just an example. Once again, if this is a file system that must be mounted, make sure it's mounted at the time the cron job runs.
Once you have done this you'll want to check that your first couple of backups have run as you expected them to. Once you verify that your system is backed up regularly, you should have peace of mind that you could restore that data should the need arise. Also, it's good form to check from time to time that nothing has gone amiss.
SummaryTo boil it all down, no matter how careful you are or how stable Linux is, chances are you will someday run into a "disastrous" crash and your only recourse will be to restore your system and data. That's why I recommend that data backups be the backbone of your disaster plan. Also, the preceding tips and tricks may be helpful in preventing a disaster. The bottom line is that I sincerely hope you never need to use the tactics outlined in this article, but if you follow these guidelines I think you'll find a Linux desktop crash won't be a disaster.
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Aug. 30, 2015 01:00 AM EDT Reads: 429
SYS-CON Events announced today that IceWarp will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. IceWarp, the leader of cloud and on-premise messaging, delivers secured email, chat, documents, conferencing and collaboration to today's mobile workforce, all in one unified interface
Aug. 30, 2015 01:00 AM EDT Reads: 382
With the proliferation of connected devices underpinning new Internet of Things systems, Brandon Schulz, Director of Luxoft IoT – Retail, will be looking at the transformation of the retail customer experience in brick and mortar stores in his session at @ThingsExpo. Questions he will address include: Will beacons drop to the wayside like QR codes, or be a proximity-based profit driver? How will the customer experience change in stores of all types when everything can be instrumented and analyzed? As an area of investment, how might a retail company move towards an innovation methodolo...
Aug. 29, 2015 11:45 PM EDT Reads: 424
The Internet of Things (IoT) is about the digitization of physical assets including sensors, devices, machines, gateways, and the network. It creates possibilities for significant value creation and new revenue generating business models via data democratization and ubiquitous analytics across IoT networks. The explosion of data in all forms in IoT requires a more robust and broader lens in order to enable smarter timely actions and better outcomes. Business operations become the key driver of IoT applications and projects. Business operations, IT, and data scientists need advanced analytics t...
Aug. 29, 2015 07:30 PM EDT Reads: 378
A producer of the first smartphones and tablets, presenter Lee M. Williams will talk about how he is now applying his experience in mobile technology to the design and development of the next generation of Environmental and Sustainability Services at ETwater. In his session at @ThingsExpo, Lee Williams, COO of ETwater, will talk about how he is now applying his experience in mobile technology to the design and development of the next generation of Environmental and Sustainability Services at ETwater.
Aug. 29, 2015 03:45 PM EDT Reads: 110
Consumer IoT applications provide data about the user that just doesn’t exist in traditional PC or mobile web applications. This rich data, or “context,” enables the highly personalized consumer experiences that characterize many consumer IoT apps. This same data is also providing brands with unprecedented insight into how their connected products are being used, while, at the same time, powering highly targeted engagement and marketing opportunities. In his session at @ThingsExpo, Nathan Treloar, President and COO of Bebaio, will explore examples of brands transforming their businesses by t...
Aug. 29, 2015 11:00 AM EDT Reads: 205
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies leverage disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 17th Cloud Expo, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advanced analytics, and DevOps to advance innovation and increase agility. Specializing in designing, imple...
Aug. 29, 2015 11:00 AM EDT Reads: 252
While many app developers are comfortable building apps for the smartphone, there is a whole new world out there. In his session at @ThingsExpo, Narayan Sainaney, Co-founder and CTO of Mojio, will discuss how the business case for connected car apps is growing and, with open platform companies having already done the heavy lifting, there really is no barrier to entry.
Aug. 29, 2015 10:30 AM EDT Reads: 115
SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.
Aug. 29, 2015 09:30 AM EDT Reads: 854
Through WebRTC, audio and video communications are being embedded more easily than ever into applications, helping carriers, enterprises and independent software vendors deliver greater functionality to their end users. With today’s business world increasingly focused on outcomes, users’ growing calls for ease of use, and businesses craving smarter, tighter integration, what’s the next step in delivering a richer, more immersive experience? That richer, more fully integrated experience comes about through a Communications Platform as a Service which allows for messaging, screen sharing, video...
Aug. 29, 2015 08:45 AM EDT Reads: 588
SYS-CON Events announced today that Micron Technology, Inc., a global leader in advanced semiconductor systems, will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Micron’s broad portfolio of high-performance memory technologies – including DRAM, NAND and NOR Flash – is the basis for solid state drives, modules, multichip packages and other system solutions. Backed by more than 35 years of technology leadership, Micron's memory solutions enable the world's most innovative computing, consumer,...
Aug. 29, 2015 08:45 AM EDT Reads: 201
As more intelligent IoT applications shift into gear, they’re merging into the ever-increasing traffic flow of the Internet. It won’t be long before we experience bottlenecks, as IoT traffic peaks during rush hours. Organizations that are unprepared will find themselves by the side of the road unable to cross back into the fast lane. As billions of new devices begin to communicate and exchange data – will your infrastructure be scalable enough to handle this new interconnected world?
Aug. 29, 2015 07:45 AM EDT Reads: 133
As more and more data is generated from a variety of connected devices, the need to get insights from this data and predict future behavior and trends is increasingly essential for businesses. Real-time stream processing is needed in a variety of different industries such as Manufacturing, Oil and Gas, Automobile, Finance, Online Retail, Smart Grids, and Healthcare. Azure Stream Analytics is a fully managed distributed stream computation service that provides low latency, scalable processing of streaming data in the cloud with an enterprise grade SLA. It features built-in integration with Azur...
Aug. 28, 2015 07:45 PM EDT Reads: 202
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome,” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Aug. 28, 2015 06:00 PM EDT Reads: 331
Akana has announced the availability of the new Akana Healthcare Solution. The API-driven solution helps healthcare organizations accelerate their transition to being secure, digitally interoperable businesses. It leverages the Health Level Seven International Fast Healthcare Interoperability Resources (HL7 FHIR) standard to enable broader business use of medical data. Akana developed the Healthcare Solution in response to healthcare businesses that want to increase electronic, multi-device access to health records while reducing operating costs and complying with government regulations.
Aug. 26, 2015 07:00 AM EDT Reads: 125
For IoT to grow as quickly as analyst firms’ project, a lot is going to fall on developers to quickly bring applications to market. But the lack of a standard development platform threatens to slow growth and make application development more time consuming and costly, much like we’ve seen in the mobile space. In his session at @ThingsExpo, Mike Weiner, Product Manager of the Omega DevCloud with KORE Telematics Inc., discussed the evolving requirements for developers as IoT matures and conducted a live demonstration of how quickly application development can happen when the need to comply wit...
Aug. 2, 2015 11:15 AM EDT Reads: 550
The Internet of Everything (IoE) brings together people, process, data and things to make networked connections more relevant and valuable than ever before – transforming information into knowledge and knowledge into wisdom. IoE creates new capabilities, richer experiences, and unprecedented opportunities to improve business and government operations, decision making and mission support capabilities.
Aug. 1, 2015 10:00 AM EDT Reads: 478
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Architect for the Internet of Things and Intelligent Systems, described how to revolutionize your archit...
Jul. 30, 2015 07:30 PM EDT Reads: 1,561
MuleSoft has announced the findings of its 2015 Connectivity Benchmark Report on the adoption and business impact of APIs. The findings suggest traditional businesses are quickly evolving into "composable enterprises" built out of hundreds of connected software services, applications and devices. Most are embracing the Internet of Things (IoT) and microservices technologies like Docker. A majority are integrating wearables, like smart watches, and more than half plan to generate revenue with APIs within the next year.
Jul. 30, 2015 02:30 PM EDT Reads: 278
Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world. Get ready to learn the facts: Is there a bias against women in the tech / developer communities? Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions? Some beginnings of what to do about it! In her Opening Keynote at 16th Cloud Expo, Sandy Carter, IBM General Manager Cloud Ecosystem and Developers, and a Social Business Evangelist, d...
Jul. 30, 2015 12:00 PM EDT Reads: 2,223