Category: Virtualization



​This seems to be the plaguing discussion among many companies today. My opinion?, there is no wrong answer, however there is alot to consider before signing up for services. I wanted to take a little bit of time to expose some of the questions you should answer before deciding to go with a hosted solution. Deciding who will handle your business critical infrastructure can be a very stressful decision to make. Many hosting companies out there will go the extra mile to sell themselves to you . But if you dont know what to ask and what to look for, you may end up locked into a contract you will be very unhappy with. There are alot of business owners out there who have a difficult time with there desktop, let alone deciding what you need for business critical applications in your company. I hope this post will be able to help all of you decide what direction to head without having to deal with sales pressure. Cause if you are like me, i have no patience for sales pressure. So lets dive in.

Some questions you should answer before starting to look for a hosted solution.

What is your application exposure?.

Is your customer base international or national?. Obviously web applications have international exposure inherently. However if you know that you have a large consumer or customer base in other countries that put alot of demand on your applications then you may need to consider more strict uptime requirements. If your application is primarily national or maybe just an internal use only, then you will have a bit more flexibility.

Unfortunately you need to consider that the more 9’s you have in your uptime, the more money you will need to be willing to spend to ensure you are able to maintain it. For many businesses, it is acceptable to have outages at any given time. Now obviously we all dont’ want to have our services to be going down frequently throughout the day, but for some it is not a big deal if your servers need a good reboot during the middle of the day.

What is your uptime requirements?. 

​This question ties into our first question and is very important to be able to answer. If your core business depends on your applications you have on the internet, then you need to look at this very closely but be realistic. Most people on the internet when they go to a web page will refresh their webpage if they get an error. so it is unrealistic to expect that 1 your servers will never go down and 2 that you can maintain 99.999% uptime with very little upfront cost. so to answer this question it would be best probably to be able to answer this. How much of your business would be impacted if your sites went down 5 minutes? then how much for 10 minutes, 15 and so on. The more time you are willing to accept. the cheaper the cost will be to maintain and keep your systems up.

why does keeping servers up cost so much?. Because you need to be able to deal with the fact that eventually your servers will go down. When they do you will need to restore backups possibly to get your data back or you will need to have another server running that is ready to take over when it goes down. if your acceptable loss of data is 5 minutes and your down time can be nore more then 3 minutes, then you will need to be making incremental backups every 5 minutes and you will also need to deploy a load balancer that will be ready to send traffic to another server if one goes down. The more data you have, the more it will cost to backup every 5 minutes. depending on the amount of data you may need to look at an enterprise type backup and restore solution to ensure you will be able to get your data back within the alotted time.

This only covers if you loose a single server. So there are many other disaster scenarios you may need to consider in this whole picture. I would suggest trying to make a list of all the possibilities you can think of, then write out how you may protect yourself against it.

Now i know this already seems to be an obvious choice to go with a hosted solution. Don’t forget though, even hosting companies have major outages that you will need to make a plan for. Look up amazon and see what kind of outages they have had. Now i am not saying they are bad, because i think they are very exceptional with there service but they like any hosting company suffer from outages. Everyone does.

How much data do you have?.

Plain and simple, the more data you have whether it is hosted or not, the more it costs to store it. We all have a fear of getting rid of data so we inherently keep everything. This is not only dangerous but expensive. Consider moving data that you don’t need right away to a tape backup or a large NAS drive that has cheap slow drives in it.

Remember it doesn’t just cost to keep data on your servers, but it also costs to back it up. Depending on how often you need to backup, you may eat up alot of space and money before you know it.

Do you have any legal or SLA requirements for backing up your data?.

Some businesses through litigation or merely through what they guarantee in their contracts need to maintain a rather large library of data. Being able to restore a file back to any point in time may be critical for you, but it costs. If you are willing to accept a file but may be from the previous day or week, then you will definitely save some money there. This may mean that you loose a little bit of data in the file but at least not all of it is lost.

what applications do you need to run?.

Some web applications are more intensive then others. Namely your databases are. These are applications that store all your dynamic data on your web page, and can require alot of CPU and memory. In hosted environments, this can get costly when considering your options.

do you need to adhere to PCI DSS, SOX, HIPAA, GLBA, FISMA compliance?.

Many hosting companies these days adhere to many of the compliance standards out there, but depending on what compliance you need to adhere to will depend on where or if you are able to use a hosting company. If you you are able to, then consider the fact that the more strict the compliance standard is the more it will cost to host your data. This is all due to the amount of security measures that need to be in place, the amount of backups provided to recover this data, and even the encryption of sesitive data that is stored.

Virtualization?

This is my favorite section, because virtualization is an awesome technology. It allows you to get more out of the hardware you own. By deploying multiple servers on one physical box, you can save yourself thousands of dollars. Now virtualization has a heavy up front cost but defintley has a high ROI. It also allows you to deal easier with disaster recovery plans, uses less power then equivalent number of servers and takes less man power to maintain the hardware. This all in turn saves money.

 

If you are able to answer the majority of these questions, then i think you should be well prepared to start conversations with hosting solutions. All hosting companies should be able to give you a cost associated with each of these questions. Most of all what you should take away from this is an old saying “Dont put all your eggs in one basket”. If you need to ensure your sites are not down for extended periods of time, consider a hybrid approach where you purchase your primary equipment to run your day to day applications and servers. Then utilize a hosting company to act as your disaster recovery site. If you want to just strictly use hosting companies to host your solution, maybe because the initial cost of purchasing your own is to much or finding the right person to support it is to daunting. Consider looking at multiple hosting providers that utilize different ISP’s incase there is ever an issue with a particular ISP. Using different hosting providers, solves some of the major news worthy outages that have plagued some of the larger hosting providers. By having redundancy in all your systems helps to ensure you that your systems will be up and ready in the event of disaster or even minor outages.

You can’t plan for everything that may happen, but the more you plan for the better you will protect yourself and your company. Hosting companies will be there to sell you what you have, hopefully this prepares you a little to know what to ask them and how they may respond. Hosting companies need there maintenance windows to upgrade or troubleshoot their hardware, be aware of these clauses and ask al about what they do in the event that they need to take down major equipmet to replace. Most of all, get it in writting.

I hope there is someone out there that this reaches and helps. I know this can be a touqh discussion to have and even think about. So dropme a comment or email and ask as much as you would like. We are here to help anyone we can as well as provide services.

Advertisements

A couple of days ago I had a sudden panic when all of a sudden my Virtual Machines began to shut down. You can only imagine how bad this can be when a fully redundant system, simply fails. As I began looking into the issue, I start to receive errors that some of my LUN’s on my brand new EVA had reached capacity.

Baffled as you can imagine I began to look at my usage in VMware and found that I still had approx 180 Gig available. When I opened up the datastore I noticed that my VM’s had a massive amount of snapshots that have piled up in the directories.

We use a utility called Backup Exec from Symantec to provide a backup utility for our entire production environment. One of the plugins for Backup Exec allows you to make backups of your VMDK’s. When BE makes backups of these files it makes a call to VMware and creates a snapshot of the VM at that moment. If for whatever reason the backup job fails, the snapshot is not deleted and it becomes the current working version of the VM. After several failed jobs this began to pile up for the VM’s and the LUN ran out of room.

Well as you remember I told you that when I looked at the LUN I had 180 Gig available. Well unknown to me that coupled with the fact that my LUN had run out of space but now VMware was not reporting my usage correctly. What this caused was a failure of the alarms triggering to tell me that my LUN utilization was getting high.

After calling VMware to assist me with getting my environment back online and clear out all my snapshots, I found the issue with the alarms not triggering. The recommendation was to edit your datastore alerts and make a change to it in some way so that when you click ok, VCenter server will reset the trigger and start to poll the actual datastore size at that moment.

After all is said and done I have learned and am now passing on to all of you. Always check your backups and make sure that there is no snapshots left behind that were not cleaned up. I have also learned to double check my usage stats and look more closely at VSphere client for anomalies.


One issue I seem to run into more and more now that I have a VMWare virtual environment, is Ghost Nic’s. yes it can be a little scary, but easily resolved. So what is a Ghost Nic?. It is when a Nic card is removed from a VM or physical box prior to removing it’s settings. So when another Nic is installed it shows as Nic #2 , and windows will complain that the IP is already in use by another Nic if you are trying to reuse the IP from the old one. Here is the odd part, there is no other Nic at least not anymore so how could this be?. Well by removing a Nic before uninstalling it, the windows registry will continue to map to a Nic that is no longer present. It holds it’s settings in the registry so that if it is ever put back in, then all your settings will show back up. Each device in windows has a unique ID that identifies it within windows and this is what windows ties the settings to.

This seems to be more of an issue now because in VMWare all your devices are now virtual to the VM, and If anything happens to your VMWare tools or you need to P2V a machine it is very easy to forget to uninstall your Nic cards before re installing or virtualizing your physical box. Of course I always forget to do this. Now if it is a physical box, the easy fix is just pop in the old Nic and uninstall the drivers and settings by going to device manager >> uninstall device then you are all set. In VMWare, well you can’t pop in a virtual Nic and uninstall the drivers because every time you install a new Nic in a VM, it is given a new UUID. So I will show you how to take care of it another way.

First goto your command prompt and run this command

Then right click on My Computer >> click on Properties >> then goto Device Manager.

Once you are in Device Manager click on the View menu option, then select “Show Hidden Devices”. You should see your ghost Nic in the list of devices in your Device Manager window. At this point right click the device, and select uninstall. Now your Nic settings are freed up and you can assign your static IP along with your other settings without having any issues.

If you have any problems, questions, comments. Feel free to leave me a comment or email me.


I just got out of 2 meetings today’s with HP and the local Cincinnati VMUG group. One of the take always on these conferences i will touch on in this post is the central theme which was mostly tiered storage and the like. I will dive into it more if everyone would like me to, however the logic makes sense.

We all know that a SAN solution is not cheap. You can easily spend several hundred thousand implementing a good solution. I know my thought at first was ok this gives me one big pool of space to use which is great, but what about the future growth of this?.

When you are in the planning phase of sizing your storage array, look at designing it in a fashion that allows you to push your least mission critical data to your slower cheaper storage, keep all your high IO important data that you need in the faster storage like SSD. You can get more bang for your buck this way while maximizing what you have.

Look at all avenues and ask questions. There is software out there built into these arrays to make them more intelligent. All you have to do is leverage it and you will be the IT hero!. Now there is a lot more to consider and I can go into more detail if you would like, but I just wanted to put this out there and hope for some feedback. Enjoy!

%d bloggers like this: