Today, Let’s talk about Storage.

I love to talk about storage.  I talk about block storage, file storage, local attached storage, shared storage and ephemeral storage,

There’s ‘Software Defined Storage’ and Enterprise Storage.  There’s Software Defined Enterprise Storage.  

Lets not forget about storage protection.  There’s RAID protection, there storage backups and there’s storage replication.

There’s storage clones and storage snapshots.  There flash storage, and there spinning disk storage.  There’s even RAM-based storage which is making a comeback in some circles.

There’s storage block IO sizes, storage interconnects such as Fiber Channel, iSCSI, Infiniband, ATA, SAS and SATA.

There’s storage strip size and there’s storage efficiency such as thin provisioning, storage compression and storage de-duplication, and there’s storage encryption ‘at rest’.

There are file systems that go on storage: FAT, FAT32, NTFS, NFS, HDFS, VMFS, EXT, EXT2, EXT3, EXT4, XFS, ZFS (just to name a few).  There are clustered file systems and network file systems and there are standalone file systems.

Did I mention – I love to talk about storage?  

The bottom line:  What is the right storage solution to meet the business needs now and into the future?, and what are the underlying characteristics of that storage solution that are actually at work to deliver the results to the business?

What are your favorite storage solutions, and why are they your favorite?  I’d like to know.  And in future articles – I’ll share some of mine, as well.

(Comments and Questions are always welcome and responded to.)

Advertisements
Posted in Uncategorized | Leave a comment

Is Information Technology departments going the way of Netflix?

I recently read an interesting article on the way Netflix transitioned to an all AWS infrastructure over the past 6 years.  (read the article yourself here: ( http://www.informationweek.com/cloud/infrastructure-as-a-service/netflix-cloud-architect-how-to-succeed-as-disruptor/d/d-id/1204327?page_number=2 )  As I was reading this article I was thinking about how simple the Netflix solution really is, and how durable the Netflix customer is to the failures of the product from time to time.

As an avid Netflix viewer I am reminded of all the times I am not able to view my desired selection, or my selection is halted in mid-stream. Yes, Netflix is like watching TV. If you don’t like what’s on – turn the channel, or come back at a later time.

I don’t know that this paradigm applies to business applications. Perhaps mission-critical business applications are more like the traditional telephone service (think 911 services). Can reliable mission-critical services be built on inherently non-reliable, non-mission critical infrastructure?

We’ve all talked about IT as a Utility – like the electric company. Just plug into the wall and begin consuming all you want. In my mind there is a difference here, too.

Remember all the times that the circuit blew and you had to reset the breaker because you used the microwave and the hairdryer at the same time? Or remember the storm knocking down trees that fell on the power lines that took down service for hours, or perhaps even days? Or what about the brown-outs that have been rolling through entire regional electric grids?

We have developed ways of coping with these problems. Like the Netflix problem we’ve learned to use different circuits for the microwave and the hairdryer. We’ve learned to buy and operate a portable generator if the threat and repercussions of being without power are too high, and there are many that are buying full home generators and solar generators to reduce the impact of regional power grids going down. We have home DSL and Cable Internet. We have wireless internet and some of us even have satellite internet. We have developed strategies for coping with outages.

I guess the real questions to ask our customers and our business leaders would be: Is our business more like Netflix, or is it more like the Telephone company? Will our customers endure Netflix style disruptions in service, or do they demand Telephone company levels of service availability?

Another key question to ask: If our customers demand Telephone company levels of service availability – Can we create applications that transcend inherent limitations of the lesser-reliable infrastructure providers while still delivering the highest levels of availability?

If we can – then we should – because that would be a powerful and valuable opportunity to build our business. If we can’t – then we shouldn’t – because that would make us look incompetent and irresponsible.

Can applications developers take commodity infrastructure that is outside of traditional IT controls, and turn it into mission-critical applications reliably delivered to our customers?

This is the same $64,000.00 question that is shaking up IT departments and executive offices all over the world. …  

Bottom Line: don’t jump into the water until you know what’s in there, and how deep it goes.  After you know those two things you can jump in and have fun if you choose.  

Reader comments and questions are always welcome and responded to.

Posted in Uncategorized | Leave a comment

Disk Queue Length @ SQLServerCentral.com

I recently posted on SQLServerCentral.com with regard to a question about Disk Queue Length…

“As an employee of a major data managment/san storage provider I get asked questions like this quite often, and I really liked the one answer: “It Depends.” This is by far the most accurate answer I can start out with in my own discussions.

As I read through the posts I think most everyone had the correct answers – for any given moment for any specific situation.

Just to clarify a misstatement…. RAID 1 or 0+1, or 10 is better suited for transaction logs than RAID 5. The write penalty of RAID 5 is (4) IO’s per write while the write penalty for RAID 1/0+1/10 is only (2) IO’s per write. Since the transaction logs are primarily write oriented, RAID 5 would cost you twice as many IO’s as RAID 1/0+1/10

The type of disk drive in the SAN array would determine the number of IOPs that could be expected out of that disk. 15K RPM Fibre Channel is usually rated around 180 IOPs per disk. 10K RPM Fibre Channel is rated around 140 IOPs, while SATA 7200 RPM is rated around 70 IOPs per disk. Solid State/Enterprise Flash Drives are similar in Write Performance to 15K Fibre Channel disks, but in READ scenarios are capable of 2500 or more READ IOPs.

In the case of the SAN Array with 10 drives (assuming they are running the highest performance spinning disk @ 15K RPM) there would be available in that array ~1800 IOPs. If the activity is all read then there is no additional RAID write penalty associated with this. This means that a query that performs a huge table scan would have about 1800 IOPs of capacity to satisfy that request. (for table scans – IO Bandwidth becomes an important factor as well, and along these lines IO READ SIZE is critical to determining MAX BANDWIDTH. This is a slightly different discussion but may be relative on TABLE SCAN intensive queries such as Microsoft’s Fast Track Data Warehouse reference architecture.)

So back to the original question where Spotlight triggers an alert at 477 IOs Queued up. – IF – the queue depth is satisfied in a timely manner – such as in a second or whatever seems reasonable to your specific environment – then this is usually considered to be fine. If the queue depth is held in an oversubscribed level for an extended period of time then this would indicate the drives are being asked to work harder and longer then they are physically capable of working, so requests for work are backing up in a queue waiting to be serviced by the disks. There are only two ways that I know of to fix long lines at the grocery store when we checkout – one would be to open up additional checkout lanes (for the SAN admin and the DBA this would mean: get more disks involved in the workload so the requests are satisfied in a timely manner), or two – make it more efficient for the checkers to process each customer more quickly – spending less time on each customer – thereby increasing their efficiency and reducing the time other customers must stand in line waiting to be checked out (translated as: the DBA would need to optimize the queries to use less disk resources – which as we all know is often times needed, but not always a viable option due to application constraints.)

Relational Database systems are probably the most complex and most difficult of all of the various systems that IT practitioners have to deal with because there is really no such thing as a “typical database”. For this reason we as DBA’s must think in terms of guidelines and principles, and best practices to direct our thinking towards the right answers for our individual situations at a specific point in time – and the only thing that we can be sure of is that things will change and we’ll have to readjust our thinking again.

I hope this helps to point you in the correct direction with regard to your database/SAN questions. I’m happy to answer additional questions. My name is Randy Loeschner and I work for EMC. I can be reached at my blog: http://www.randyloeschner.com. Cheers! “

Posted in Uncategorized | 1 Comment

Hello world!

Yes, this is my blogging birthday.  Today I am born into the blogosphere – and NO, this is not my real birthday – though gift certificates to Dairy Queen are always accepted.   The irony of the “hello world!” is not lost on me.  This is the first code we are all taught to write when we learn a new programming language.  Certainly the language of the blog will be just as new and just as unique.  So here we go…. Hello World!  What shall we talk about today?

Posted in Uncategorized | Leave a comment