|By Mark R. Hinkle||
|August 31, 2004 12:00 AM EDT||
It's 2:00 a.m., you're working on that critical presentation, and the power goes out. Since you moved your power supply to your significant other's computer, you just lost all your work.
We've all been there one time or another. Then the real trouble starts: not only haven't you saved your work in an hour but lo and behold your PC won't boot back into your operating system. As the cold sweat drips from your brow you realize that in addition to losing your presentation, you've also lost your financial records, calendar, and more data than you could ever hope to replace. Before it happens to you again you need to have a desktop backup strategy.
This scenario resonates with many, if not all of us. It's not an individual problem; it's a computer-user problem common among suits and "propeller-heads" alike. That's why I'm focusing on ways to avoid desktop disasters this month.
In addition to focusing on backup disasters I'm going to delve a little more deeply into the command line to solve some of these problems. I don't want to intimidate anyone who's more comfortable in the GUI world, but there are many easy-to-use and powerful command-line utilities that are at your disposal in most Linux distributions. Also, I have been recently inspired by Doc Searls (http://doc.weblogs.com) and his movement for DIY (do-it-yourself) IT; check out his IT Garage (http://garage.docsearls.com/). This idea isn't fascinating or interesting to me because it's new; it's because I come from the do-it-yourself generation, the one that makes it possible for a DIY channel on cable and 101 home improvement shows to exist. What's interesting and noteworthy is that, with a little knowledge, you can save yourself a lot of money by doing some of these tasks yourself. So with a nod to my fellow do-it-yourselfers, let's explore the world of Linux desktop backup strategies.
Desktop Backup StrategiesThere's an old saying, "An ounce of prevention is worth a pound of cure." This phrase could just as easily have been attributed to a system administrator after a server crash if it wasn't first attributed to Ben Franklin. More often than not we as computer users have not taken the most basic precautions with our personal computing environments. The bottom line is that there is no foolproof way to keep our PCs from crashing, so backups are critical. Please follow me as we take a walk down backup lane.
DataThe first step in developing a backup strategy for your data is to figure out where your data lives on your system. By default almost all your user data is stored in your /home/$USER directory ($USER just means the user you log in to Linux with). Once you confirm that this is where your information is, you can decide how you will make backups. In my case, I have a DVD/RW on my laptop. I try to keep the bulk of my data in one directory but it's actually a fat32 partition that I access from Linux or Windows. That directory is /windows/D/. I simply launch my favorite CD burner software k3b (www.k3b.org) and create a data project to copy all my critical files to a DVD or CD. Its drag-and-drop interface allows me to make copies easily, but that's only one of the many ways to archive your data. You can also copy to a second hard drive or other storage device or even back up your data over the network. However, the method is not nearly as important as remembering to do it, or at least scheduling the system to do it on a regular basis (see the sidebar How to Automate Backups).
Operating System BackupsMost of you who have read my previous columns will know I'm a Linux LiveCD junkie (www.linuxworld.com/story/45259.htm); Linux LiveCDs can be very valuable when backing up your operating system. One method for creating backups is to use the handy dandy Knoppix CD (www.knoppix.org) and then do a complete disk copy to an extra hard drive (see Figure 1). Now this is not the fastest way to do things, but it catches everything on my hard drive (including all my data) and it's very easy to verify the contents of the backup. In this scenario I'm going to use the example of a PC with two hard drives. The first hard drive will be where your operating system resides and the second hard drive is where we will store your data. Not everyone will have a second hard drive but in these days of cheap storage it's not cost prohibitive to have a second internal or an external hard drive. For a 60GB hard drive there are many options well under $100. Since you don't have to invest in backup software with these techniques, you'll most likely have the extra cash.
How to Back Up Your Hard Drive Using KnoppixYou can create backups of your hard drive in a variety of ways; however, copying the entire contents of one local drive to another can create the most thorough backup. To start, boot your computer using your Knoppix CD. You should see both hard drives on your desktop and if you open your shell you can sudo su to gain root (super-user) access. You can also discern which drives are which by their contents, which should be browsable using the Konqueror file manager. Make sure that once you have discerned which hard drive is which by browsing them that you unmount them so you can copy one drive to another. One drive is likely to be /dev/hda and the other is probably /dev/hdb. You may need to research them a little more thoroughly to be sure. To copy one drive to another simply type the following command:
dd if=/dev/hda of=/dev/hdb
Imaging Your Drive from Another ServerYou may not have room for a hard-drive image on your local machine and may choose to keep it on a server so that you can migrate it to newer hardware or use it for other PCs in your enterprise. Once again you can use the same dd command in conjunction with a variety of remote access protocols to reimage your disk. While I have done this a few times, I defer to an expert on this topic, J.H. Moore, who has put together an excellent how-to at www.okmoore.com/imagedrive.html, mirrored at knoppix.net/docs/index.php/ImageYourHardDriveUsingKnoppix.
Give Your Hard Drive a KickStartInevitably you'll have to reinstall your operating system on your desktop Linux PC; in a larger environment you may have to do it for many machines. Part of a good disaster plan is a fast recovery, that's why you may want to use Red Hat's KickStart to automate the reinstall or even the initial reinstall of systems. The process is to build one PC that has your preferred configuration (this is called the build machine), then use that template to "kick off" additional installs. This automates the installs and minimizes human interference (and consequent mistakes). This is a good measure for fixing failed machines because you basically maintain one master machine and then let it configure your additional PCs painlessly. This tool was designed for use with Red Hat Linux, but many people have hacked it for other distributions. A search on Google for "Kickstart Linux" will yield a bounty of information on the subject. For more information, check out the Kickstart (listman.redhat.com/archives/kickstart-list/) mailing list as it's a good place to ask questions and get ideas on what other Linux users are doing with this tool.
Other Tips and TricksBackups are obviously essential to recover from a desktop disaster, but here are some tips to prevent you from having to use your backups or fix your installation before you resort to a restore of your system.
Protecting Files: Making Data Read OnlySometimes we have data that we store but never alter. In those cases it could be advantageous to make the data read only so it doesn't get overwritten or deleted. You can make your data read only by using Linux permissions. Every file and device on the system is controlled by permissions, and before a file is executed, written, or read there it checks to make sure that that action is allowed.
Users and GroupsEvery process on a Linux system is executed by a user from the super-user root to a user with restricted permissions, so he or she can't compromise other parts of the operating system; in my case, this user is mrhinkle. Each of these users is part of one or more groups, which makes it easier to share resources and still enforce permissions. All information about users and groups is kept in /etc/passwd and /etc/group/. Now if you do have data and you don't write it often, or maybe it's just an archived copy of a presentation, you could change the permissions for the files to be "read only." That way it's harder, but not impossible, to be overwritten or deleted. Do this by using the chmod command. Chmod sets permissions for a file. The syntax is typically chmod ### file where ### are the permissions for owner, group, and all other users. For fields that I don't want to get overwritten I occasionally set that number to 444, where the four indicates read only.
In the example below I've created the file example.txt using the vi editor. Once I created the file I used chmod to set the files to read only so that I don't overwrite the data there. Notice that when I try to remove the file I also get warned. This isn't a foolproof method and I could have answered "y" to the rm dialog and the file would be deleted, but the warning should help me to think twice before I edit the file.
You may also want to be extra cautious and have the files owned by another user with just read-only attributes. For example, if I normally work as mrhinkle, I could have the files owned by root. That way, the only way I could delete them would be if I was logged in as root. You may even want to create an archive user, then use the shown command to "change owner" to archive. Keep in mind that to do this you must have permissions for that file so it's easily done as the root user. The syntax for this would be:
[email protected]:~> chown archive example.txt
Keep in mind that I first created the user archive, then changed the ownership of my files.
Creating a Boot FloppyMost distributions offer various utilities for creating a boot floppy in case a misapplied kernel update or other disaster strikes. If you want to find out how to make your own, you can try this method. It takes your kernel image and copies it to a floppy disk.
Step 1: Find the KernelFor the most part, your kernel is usually going to be in /vmlinuz or /boot/vmlinuz (on my SuSE 9.1 installation it's /boot/vmlinuz). This is a soft link to my kernel, which is vmlinuz-2.6.5-7.95-smp.
Step 2: Copy the Kernel to a FloppyYou can do this by copying the kernel image to your floppy; in most cases this will be /dev/fd0.
dd if=/vmlinuz of=/dev/fd0
Step 3: Set the kernel image on the floppy to the location of your root system.rdev /dev/fd0 /dev/hda7
Your root filesystem may be some-where other than "/dev/hda7". You might find that your Linux installation is somewhere other than "/dev/hda7". I found out where my Linux installation was by changing to the root user and using the fdisk utility tool, and listed my partition table.
Simple Backup ScriptMost of us are used to our point-and-click environments, but sometimes it's easier to use the command line. However, if you're like me, you forget the syntax to the commands and make mistakes that cause the commands to become troublesome. That's why I took a little time to find a simple script to help you create a compressed archive directory complete with date. It also requires that you write a shell script. I chose to write the following script I called "arcive":
Step 1: In your favorite text editor type the following three lines:
tar czvf $1.$(date +%Y%m%d).tgz $1
Step 2: Save the file and make sure it's executable. I saved my file and called it arcive. (Since I have some other files called archive I decided to change the spelling so I wouldn't be confused. You could call it whatever you want.) I had to change the file to make it executable by using the chmod command:
chmod 755 arcive
Step 3: Now the easy part: I'm going to make a backup of my Firefox Web browser folder so I can preserve all my bookmarks and plug-ins as well as copy them to another test PC. The format for doing this is to simply enter:
where directory name is the name of the directory you want to archive. Since I archived a directory called Firefox I got the following result:
Notice that the name has the date (July 12, 2004) in it so I can easily track when I made the backup. This is a simple script that I find very useful as a noncommand-line guy; it makes it easy to archive directories.
How to Automate BackupsThere are as many ways to back up your data as there are types of data. Here's a quick way to back up your data that's mirrored in the exact same format that it's stored in your home directory. Keep in mind, this is from a command-line interface. You'll be using two commands: rsynch and crontab. They're probably installed on your system already, but if not you may want to use Google to search for more information on their installation and use.
Rsynch is used to synchronize with files at another location. In this simple example, we'll be synching to another directory on the hard drive. In this example I will be synching my data from my /home/mrhinkle/data/ directory to a drive I have mounted at /mnt/backups/.
Cron is a daemon that executes scheduled commands. In this case, we're using it to schedule backups, but you could use it to schedule the upload of files to a Web site, to archive mail, or for a variety of other tasks.
To start this exercise, type the following at a shell prompt:
then add a line in the format below. Keep in mind that the anatomy of a cron file looks like this:
30 4 * * * rsync -a /home/mrhinkle/data/ /mnt/backups/
30 4 * * *
This is the line that indicates the time at which to run the commands further down the line. Remember that your PC needs to be turned on at this time for it to run. (This should be evident but sometimes we forget the basics when we enter new territory.) The first number is the minutes; the second the hour; the next three asterisks indicate day of the month, month of the year, and then the days of the week (acceptable values here are 0-6 with zero equal to Sunday through Saturday. I used the wildcard "*" to indicate every day of the month, month of the year, and day of the week).
This is the command. Run rsynch, which will copy the data from one directory to another and only synchronize the data that has changed since the last synch. In the case of this example I put all my data in one directory. Other things you may want to back up are bookmarks and browser settings. (I use Firefox so all my browser preferences are saved in /home/mrhinkle/.mozilla/.)
This is the data I want to synchronize. In this case the data is stored in /home/mrhinkle/data/.
This is the location I want to synchronize to. In a desktop PC this might be a second hard drive or ideally a remote file server to ensure additional redundancy. I have many different configurations so this is just an example. Once again, if this is a file system that must be mounted, make sure it's mounted at the time the cron job runs.
Once you have done this you'll want to check that your first couple of backups have run as you expected them to. Once you verify that your system is backed up regularly, you should have peace of mind that you could restore that data should the need arise. Also, it's good form to check from time to time that nothing has gone amiss.
SummaryTo boil it all down, no matter how careful you are or how stable Linux is, chances are you will someday run into a "disastrous" crash and your only recourse will be to restore your system and data. That's why I recommend that data backups be the backbone of your disaster plan. Also, the preceding tips and tricks may be helpful in preventing a disaster. The bottom line is that I sincerely hope you never need to use the tactics outlined in this article, but if you follow these guidelines I think you'll find a Linux desktop crash won't be a disaster.
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
Aug. 30, 2016 10:30 AM EDT Reads: 374
Data is an unusual currency; it is not restricted by the same transactional limitations as money or people. In fact, the more that you leverage your data across multiple business use cases, the more valuable it becomes to the organization. And the same can be said about the organization’s analytics. In his session at 19th Cloud Expo, Bill Schmarzo, CTO for the Big Data Practice at EMC, will introduce a methodology for capturing, enriching and sharing data (and analytics) across the organizati...
Aug. 30, 2016 09:08 AM EDT Reads: 167
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
Aug. 30, 2016 03:45 AM EDT Reads: 513
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
Aug. 30, 2016 02:30 AM EDT Reads: 1,861
The emerging Internet of Everything creates tremendous new opportunities for customer engagement and business model innovation. However, enterprises must overcome a number of critical challenges to bring these new solutions to market. In his session at @ThingsExpo, Michael Martin, CTO/CIO at nfrastructure, outlined these key challenges and recommended approaches for overcoming them to achieve speed and agility in the design, development and implementation of Internet of Everything solutions wi...
Aug. 30, 2016 02:00 AM EDT Reads: 2,248
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
Aug. 30, 2016 01:30 AM EDT Reads: 3,053
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
Aug. 30, 2016 01:00 AM EDT Reads: 1,941
Smart Cities are here to stay, but for their promise to be delivered, the data they produce must not be put in new siloes. In his session at @ThingsExpo, Mathias Herberts, Co-founder and CTO of Cityzen Data, will deep dive into best practices that will ensure a successful smart city journey.
Aug. 30, 2016 12:00 AM EDT Reads: 1,678
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Aug. 29, 2016 10:00 PM EDT Reads: 2,496
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
Aug. 29, 2016 08:30 PM EDT Reads: 2,452
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Aug. 29, 2016 07:00 PM EDT Reads: 1,989
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
Aug. 29, 2016 02:15 PM EDT Reads: 3,751
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Aug. 29, 2016 12:45 PM EDT Reads: 2,043
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs a...
Aug. 29, 2016 12:15 PM EDT Reads: 887
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Aug. 29, 2016 12:00 PM EDT Reads: 3,210
SYS-CON Events announced today Telecom Reseller has been named “Media Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
Aug. 29, 2016 08:00 AM EDT Reads: 989
Pulzze Systems was happy to participate in such a premier event and thankful to be receiving the winning investment and global network support from G-Startup Worldwide. It is an exciting time for Pulzze to showcase the effectiveness of innovative technologies and enable them to make the world smarter and better. The reputable contest is held to identify promising startups around the globe that are assured to change the world through their innovative products and disruptive technologies. There w...
Aug. 29, 2016 07:30 AM EDT Reads: 843
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Aug. 28, 2016 10:30 PM EDT Reads: 4,089
I wanted to gather all of my Internet of Things (IOT) blogs into a single blog (that I could later use with my University of San Francisco (USF) Big Data “MBA” course). However as I started to pull these blogs together, I realized that my IOT discussion lacked a vision; it lacked an end point towards which an organization could drive their IOT envisioning, proof of value, app dev, data engineering and data science efforts. And I think that the IOT end point is really quite simple…
Aug. 27, 2016 12:45 PM EDT Reads: 2,417
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abil...
Aug. 27, 2016 02:30 AM EDT Reads: 2,111