And That Wraps It (Cisco Live! ‘09) Up!

So, it’s over.  I’m enjoying my last night in San Fran.  Went up to Fisherman’s Wharf (tourist hell, as I call it) and had some crab…..  Now I’m sitting at a Starbucks with a great view of the street so I can people watch.  That’s my one complaint about San Francisco….everything down here closes at 9.  There are some great coffee shops near here but they all closed 2 hours ago.  Oh well….

More good sessions today.  Brain overload at this point, but good points to take back with me and work on further.

Cisco Nexus 7000 Switch Architecture

Good in-depth discussion on what makes the 7000 tick and the possible hardware configurations and their pros and cons.  Near the end we got in to some real detail that I’m not sure I’ll need, but it’s never a bad thing to have more information than you need.

Troubleshooting the Unified Fabric (FCoE)

This session focused on the Nexus 5000 and 2000, since they are currently the only models that handle FCoE.  Well, the 2K doesn’t but it acts as an extension to the 5K.  Anyway…  We covered the processes and tools for debugging L2 data traffic and FCoE across the switch.  The trick on these is that you really have several different platforms to manage at once.  There is a mini-MDS, basically, in the expansion slot (if you are doing FCoE), then that FC traffic gets encapsulated in to FCoE, and on top of that you also have standard data traffic.  Different traffic requires different tools.  Also covered were standard debug commands and outputs as well as packet output for looking at things such as DCBX (Data Center Bridging eXchange Protocol) in case the automated feature enumeration isn’t…well…so automatic.

Securing the Virtualized Data Center Environment

This session covered security end-to-end across the entire DC when you use virtualization.  Most people think of virtualization as just simple VMware server virtualization but forget things like storage virtualization, device virtualization (think virtual firewalls), etc.  So this session was a good overview of everything to consider.  How can you use VSANs to segment your storage fabric for the sake of security?  What are the implications of using virtual firewalls in an ASA?  Do you want to use VDC (Virtual Device Contexts…virtual switches in a Nexus) to segment your network?  Very good session.

Conclusion

The conference was great.  This was my first Cisco Live/Networkers and I hope to be back next year in Vegas.  It’s really hard to get to everything you want and most cases it’s simply impossible due to scheduling.  Cisco gets it, and I’m happy to see that.  They aren’t just stamping “virtualization” on rehashed products hoping to sell more gear.  They are really creating things that help you get the job done and make your environment more productive.  I think with the release of vSphere and the Nexus 1000v we’ll see a great deal momentum build.

Is It Still Too Early for FCoE?

There has been some good chatter going around concerning FCoE (Fibre Channel over Ethernet) from both Scott Lowe and Chad Sakac.  I had a chance to talk with Chad and Stuart Miniman today and I pretty much agree (go figure) with their thoughts, which align with Chad’s post.

For Varrow, in this context, we really see two types of customers.  On the smaller side we have customers buying their first SAN.  Even without FCoE they almost always choose iSCSI.  It’s easy.  It fits their environment.  It doesn’t require the knowledge and capital to implement like a Fibre Channel fabric.  Even some of the more mid-sized customers are choosing iSCSI over FC when moving up to newer storage arrays, especially if they are on older FC gear and need to refresh.  The majority of their applications just don’t require the throughput of Fibre Channel, contrary to their initial beliefs in many cases.

Larger customers aren’t starting over with a new fabric and throwing out the old.  They are either continuing on with their existing FC fabric and possibly expanding, or they are looking to supplement it with FCoE and eventually make a full migration.  The “issue” of single-hop limitations and no native FCoE support in EMC arrays hasn’t been an issue.  They put in a gateway type solution such as the Nexus 5000, for other benefits usually, and connect the legacy fabric to that and the new gear gets to use FCoE.  As FCoE matures and people become more comfortable with it we’ll see more large scale deployments, and by then these issues will be a memory.

And Now for Day 3 of Cisco Live!

So day 3 is in the books.  It was pretty much a continuation of day two with some very good sessions.  Some better than expected, others not.

Deploying Virtual Desktop Infrastructure (VDI)

This session started out pretty slow with a VDI 101 review.  Sometimes I have to remember I’m at a Cisco conference and not everyone is dealing with virtualization every day like I am.  By the time it finished I found it very informative.  The presenter had some very good statistics and test results comparing the different remote protocols (RDP, ICA, and ALP), how they handle different content, and how devices such as Cisco WAAS can make them even better.  We also covered some architectures and recommendations for deploying these devices to increase scalability and reduce bandwidth to clients.  Other topics and optimizations were covered such as print jobs and video streaming.

End-to-End Data Center Virtualization

Honestly, I wasn’t expecting a lot from this session except maybe a rehash of many things already covered in more detail in other sessions.  In fact, I had planned to duck out of this one early to go meet with Chad Sakac and Stuart Miniman for a few minutes at the EMC booth.  I didn’t get there on time….

A huge amount of information in this session…I mean, like 120 slides of network diagrams, protocol stacks, suggested architectures, etc.  How we got through what we did in 2 hours is amazing.  This was a complete walk through of Cisco’s vision for a truly virtualized data center.  Not just VMware on servers, but everything and how you deploy it, connect it, and manage it.  It took everything from other sessions and put it all together.  By far the best session I’ve had and it will take a lot to beat this tomorrow.

Super Session: Data Center Virtualization Architectures – Road to Cloud Computing

Not a whole lot to this session.  Virtualization Architectures?  Not really…  It was an executive overview of Cisco’s virtualization offerings like UCS and Nexus.  Nothing new here if you’ve been keeping track.  Yesterday’s Intel session was more informative.  Then again, I have to remember that not everyone is up on the latest virutalization offerings here.  There are a ton of attendees focused on routing, switching, voice, and WiFi.  In fact, I’m a pretty small minority at this conference.

Panel – Next Generation Data Center

Good panel event.  I like being on panels during an event and attending them as well.  Sometimes it’s nice to not be on the spot and I get to ask questions.  Great panelists for this one including representatives from EMC, NetApp, VMware, Cisco, Panduit, APC, and Oracle.  Most of the questions were directed toward the cloud computing concept, what it takes to get there, and the obstacles people face.

Almost Over…

Tomorrow is the last day of the conference.  My day, again, focuses on virtualization and the data center.  My first session is on the Nexus 7000 architecture and should make my head hurt at 8am.

Cisco Live, the Second Day

Today started early with the first session at 7:30am.  That’s one case where a bit of jet lag helps out, it didn’t help out when I got back to the hotel at close to midnight.  The sessions today were more in-depth and packed with information.

Fibre Channel Storage Area Network Design

This was a lot of information for 7:30am!  I’ll admit, I need to learn more about FC deployment and design and this was a great class.  I learned a lot, and would really like to sit this one again so I’ll be studying the slides and follow-up information they gave out.

Deploying Nexus 7000 in Data Center Networks

Another great session.  This covered the design and deployment of the 7000 and how it integrates in to existing network architectures.  Other areas such as security, best practices, and common questions were covered.  The presenter also walked through several examples of how the configuration CLI compares to that of existing Cisco 6500 systems.

Intel Super Session

The Super Sessions are normally showcases for large partners and this was no exception, but it had more interesting information than I had expected.  They covered the new Nehalem processor architecture and showed the ROI for replacing old single core systems.  They also demonstrated Flex Migration where you can VMotion from an older CPU architecture to a new server with the Intel 5500 Xeon platform.  Intel has a heavy ocus on mobile devices and covered their WiFi/WiMax testing.  Finally, they gave their vision for 10Gb connectivity and adding virtualization technology in to the hardware.

Security and Virtualization in the Data Center

This session covered infrastructure security starting from the Core and working out to the virtualized servers.  A point stressed that I really believe in:  Security requirements shouldn’t change with virtualization.  Also, don’t do things in the virtual world you wouldn’t do in physical.  For example, if two VMs have very different security profiles don’t run them on the same vSwitch or same VLAN.  Sometimes people get sloppy.  There was pretty heavy discussion on the Nexus features dealing with security, such as CoPP, broadcast suppression, Packet Sanity Checks, and LinkSec.  Real good designs for using the VDC (Virtual Device Context) in the Nexus 7000 showing how to split a single device in to multiple and use them for separate purposes without the nead for multiple physical switches.

The Rest

After the sessions I spent more time in the exhibit hall, mainly talking to the guys in the Intel booth about FCoE and their 10Gb features such as Virtual Machine Device Queues (VMDq), which offloads packet sorting and queuing to the Intel NIC speeding up transfers and reducing CPU overhead.  I’m Twittering granular updates here.

VMware and Cisco Demoing VMotion Between Data Centers

While at Cisco Live I stopped by the VMware booth to check out the proof-of-concept that VMware and Cisco are currently doing involving extended distance VMotion.  VMware’s latest information about it is here.  The idea behind this is that you can distribute your CPU resources in multiple sites.  This should be especially useful for those running out of space, power, and/or cooling in existing data centers.  Now you can split your infrastructure without losing the benefits of virtualization.

The great thing about this is that it doesn’t require anything special.  No magical network hardware.  No software updates or changes.  So what are they testing?  Simply, they are putting parameters and processes around the implementation and support.  How far is too far?  How much bandwidth do you need?  What is the impact on an application when it is cut over?  Looking at the demonstration that VMware is showing and the results from the initial testing it appears that the impact will be minimal.  The switchover time is only a little bit longer at 80KM than it was with the servers sitting in the same room.  They also went further to test the impact on a SQL server by having the VM in one location and the storage in another, connected by FCIP.  The degradation was minimal, just a few percent, if even that.

I’m curious to see what the final supported distances and bandwidth requirements will be.  VMware is saying they’ll be giving a lot more in-depth information on this in August and hopefully fill in some of those missing pieces.

The Fun Begins at Cisco Live!

Well, today was my first real day at Cisco Live…or Networkers..or whatever you call it.  Let me just say one thing.  For a guy used to North Carolina weather in June, it’s cold in San Francisco!  I almost needed a jacket tonight.  Crazy!

Had a couple of good sessions today.  Nothing too intense, that starts in the morning, but informative nonetheless.

IT Management Kickoff – Navigating the Downturn, Preparing for Upturn

This was a session on managing IT now in the downturn and how to prepare for the upturn in the economy.  Interesting ideas and some good points.  For example, I don’t think many organizations are worried about employees on the upturn.  Will your employees stay?  Will you have brain drain?  Some information that’s pretty straight forward such as focusing on ROI, and in most cases it’s a very short ROI.  Another good point is how to leverage IT in the upturn and what users expect now.  College grads go in to the workforce used to information at their fingertips, can your organization give them that?  Finally, an interesting statistic in that the average span of a CIO in an organization is 2.5 years.  Makes selling those long term strategies tough!  Forget projects that span an extended amount of time.

Realizing Benefits of Unified Networking – FCoE – By Intel

This was more of a case study by Intel about Intel and how they moved to Unified Networking/Consolidated I/O with FCoE.  They also discussed their 10Gb hardware offerings.  If you’ve been following this space there wasn’t really any earth shattering news there.  They did throw out some interesting ROI/TCO facts, mainly that they are getting the performance of Fibre Channel with FCoE but at a 25% reduced cost.  This is saving them almost $38K per rack of equipment.  Interesting.

The day ended with the reception in the exhibit hall.  Good beer selection, food looked good (I didn’t partake in that..just the beer!).  Seeing as we, Varrow, are not a “true” networking company I spent the better part of my time at the vendors in the data center aspects we are focused on.  You see the Nexus 1000v everywhere.  The entire Nexus platform is really gaining steam.  I think people are starting to “get it” and what it can do for them.  VMware is demoing their new extended VMotion between data centers.  There is no real magic here.  No new hardware.  No new software.  They are just pushing the current VMotion technology to its limits and seeing how it performs.  When all this testing is done they’ll be releasing some updated requirements for just how far you can go.  The demo they are showing in their booth is an 80KM distance between two data centers.

Today was a good start to the conference.  Tomorrow is going to be something else.  I’m starting at 7:30am and going until the evening.  The longest break between sessions is 30 minutes, so hopefully I can grab a bite somewhere in there.

Hello from San Francisco!

Well, it’s time for Cisco Live.  I’m sitting in the lobby of the Hilton people watching and finalizing my session schedule for the week.  I spent some time this afternoon exploring downtown San Francisco.  I’ve been all over southern California, but never San Fran so it’s been fun.

As for what I’m attending this week, it’s what you’d expect, heavy on Nexus, virtualization, and consolidated I/O.  There are some great sessions this week and I’m really looking forward to it.  I’ll be blogging and Twittering from the conference all week.  My user on Twitter is nash_j.  I’m just getting in to Twitter so be kind!  Tomorrow is mostly lab sessions and the exhibition hall with things really getting going on Tuesday morning.

Saving Power with vSphere

Last week I wrote a bit about vSphere’s Data Recovery feature.  There are a lot of good things to talk about in vSphere so this week I thought I’d cover Dynamic Power Management, or DPM.  Many users of VI3 are probably familiar with DPM.  It was available, but not fully supported since it was marked Experimental.  Since it was Experimental it wasn’t commonly implemented, though it was the topic of much conversation.

Dynamic Power Management allows vCenter to put ESX servers in a standby state when they aren’t needed as a way to reduce power consumption.  If you have used Dynamic Resource Scheduling, DRS, in VMware you are familiar with many of the concepts that DPM uses.  With DRS when load is unbalanced vCenter will vMotion virtual machines around in the cluster as needed to balance the load across the servers.  To save power the DPM functionality monitors load on the cluster as well and when possible removes all virtual machines from an ESX server and then puts that server in standby.  Instead of going through the work of creating a demonstration I’ll link to a great video already done showing DPM in a shifting workload environment.

• DPM Video Demonstration

Putting DPM to Use

The configuration of DPM closely resembles that of DRS.  As with DRS you set an automation level.  The values are:

• Off – Disable DPM and do not provide recommendations
• Manual – Provide recommendations but do not carry out actions automatically
• Automatic – Automatically execute on recommendations, if the VMs can be moved automatically

Look familiar?  I thought so.  By default all hosts in the cluster inherit the cluster’s DPM automation level but this can be overridden on each host.  Again, similar to DRS, there is also a slider threshold to set the priorities of the recommendations from priority-one (required action) to priority-five (slight improvement).  By adjusting the slider you can change the sensitivity so that only priority-one recommendations are executed all the way to priority-five.

Hardware Requirements

DPM will put an ESX host in a standby state when its compute resources are not currently needed.  To bring the host out of that state it must use one of the three supported wake-up technologies.  These are:

• Intelligent Platform Management Interface (IPMI)
• Hewlett-Packard Integrated Lights-Out (iLO)
• Wake-on-LAN (WoL)

If a host supports multiple protocols, they are used in the following order: IPMI, iLO, WoL.

How Much Can I Save?

How much you save really depends on your environment and the hardware you are using.  From an earlier post, do you know what your data center costs to operate? To give you some idea let’s take a look at a server that uses 1KW/h of power.  That’s a lot, but I like round numbers.  If you can put that server in standby for 8 hours per day that can save you $584/year, including cooling costs.  Scale that by the number of servers that run needlessly over nights and weekends and you can really see the benefit of this feature.

Hey Wait, is This a Good Idea?

So, do you want an automated system powering down your servers?  That’s a topic of some debate.  For some opinions:

• VirtualGeek
• Chris Wolf
• Mike Laverick

On the technology side you have to put some faith in to those magic WoL packets to start your servers up again.  What happens when that doesn’t work on Monday morning?  That’s where good testing comes in.  On the policy side will most organizations let an automated system make these sort of changes or does each server cycle require invoking the change control process?

I think that in time this will be common place.  It’s going to be driven by need as many data centers and server rooms are forced to find ways to conserve power and reduce operational costs.  Like most technologies it will come with time, testing, and comfort.

About vSphere’s Data Recovery

With the release of VMware’s vSphere 4.0 there comes some great features and additions.  One that hasn’t gotten as much attention as others is the Data Recovery feature.  While FT (Fault Tolerance) is cool and flashy, Data Recovery is probably useful to more people, especially in smaller shops.

So, what is Data Recovery?  It’s the new simplified method to backup and restore virtual machines.  It’s not a replacement for standard backup systems but can be used in place of add-ons such as vRanger Pro and Veeam Backup, assuming you don’t need some of the advanced functions they provide.  Basically, it creates a backup copy of your selected VMs to another data store or CIFS server.  From that location you can then back up those VM files to tape, other disk, whatever your backup system happens to be.

Data Recovery Architecture

Data Recovery is implemented as an add-on to VCenter and uses a virtual appliance.

Destination storage can be any supported storage such as iSCSI, Fibre Channel, or NFS as well as a CIFS share.  One of the best features of this system is that it employs integrated deduplication to save storage space.  Be sure to store backups of VMs with similar data in the same location.  For example, if you have 10 servers running Windows 2008 Server be sure those are backed up to the same location so that you only store one copy of the Windows 2008 Server files.  The deduplication happens in-line as the VMs are being backed up, not once they are on the destination storage.  This is a big deal since the latter would require more space to hold the data until deduplication happened but since the process is in-line that isn’t the case.

Backup & Restore

Backups and restores are done via the GUI interface.  Just select the VMs you want to backup and the destination storage you want to use.  VMs are backed up via a snapshot.  Windows systems can take advantage of VSS to get application consistent copies.

One thing to note is that the only option for backups is the whole VM.  You can’t selectively pick certain files and directories, it’s all or nothing.  That is where the data deduplication is appreciated.

Restores are pretty much the reverse, but one bright spot is that you can restore an individual file.  Please note that this feature is experimental and not fully supported.

Thoughts

Data Recovery is a simple way to handle backups of your VMs, but is it for everyone?  For one, VMware says that Data Recovery is good for up to 100 VMs.  So if you’re environment is larger than that it may not be for you.  Also, if you employ other backup systems that plug directly in to VMware what you are using now may be a better alternative.  We have many clients using vRanger Pro and Veeam Backup with great success.  Will those types of customers continue to need a third-party application?  That remains to be seen but I think many people will use Data Recovery.  It’s included in most licensing options and integrates directly into vCenter.  Unless they need some advanced functionality in the other apps they may decide to forgo the expense and just use Data Recovery.

Useful links:

• Data Recovery FAQ
• Demonstration Video (Pre-release)

Can I Virtualize My VCenter Server?

Yes.  Well, thanks for stopping by!

Okay, not so fast.  We get this question a lot, I mean, a LOT.  It’s a valid question.  Must a customer dedicated a physical server just for VCenter?  There are pros and cons for either way.  It’s really up to the client to decide.  Our standard is to ask for a physical server for VC, but we don’t always get our wish.

Why wouldn’t you want to virtualize VCenter?  When everything is working fine there really is no reason not to virtualize VC.  But, the problem is what happens when things don’t go well.  Do you really want your management platform to be tied to the system that it is managing?  Consider these scenarios:

• ESX will continue to function two weeks after losing the license server so it’s rare that VC would be down that long, but what happens if that does happen?  The VCenter VM is down and the license has expired.  You can’t power up the VC server because, again, your grace period is up.  You get to rebuild VCenter.
• Maintenance can be a problem since VCenter is required for VMotion.  If the VCenter VM is down for any reasion you won’t be able to seamlessly move VMs to other hosts.
• Need to resignature LUNs, but VCenter is on that LUN?  Can be a problem.

For most of these they would be rare events, but in these circumstances a rare event could turn in to a lengthy headache.  This is why the best practice is to put VCenter on a physical server.