Well played, Microsoft

I wanted to cancel my account before I got hit up for “hidden charges” or something ridonkulous like that and could not find any way to cancel my subscription short of calling someone on the phone which could be my least favorite thing to do.

From what I could tell, I should be allowed to “opt-out” of an automatic renewal, except I don’t appear to have that option. I am wondering if I am simply not getting a proper user experience because I am part of some trial program, but it is rather annoying to not be able to simply click a button and say “stop charging me at the end of this month”.

As of right now I owe $1.06 for usage charges over the past month. I would be fine with that except for the fact that I was under the impression I was not going to be charged anything at all during my trial period. I find it hard to believe that Microsoft needs my $1.06 that badly but apparently they do. And they also need it again next month, so that may be why they won’t let me cancel my account so easily.

I suppose I could just drop the server I am using, essentially ending any chance of them charging me for SQL Azure altogether, but something tells me they would find a way to keep billing my credit card. So instead I will just open a new credit card, change my billing information to use the new card, and then just cancel the card.

That should take care of them trying to charge me for a trial service they don’t want me to cancel easily. And yeah, I understand that opening the card and closing it is a lot more effort than just calling someone at Azure support, but I am annoyed to the point that I would go out of my way to circumvent the “process” they have configured.

I still think the biggest hurdle will be how Microsoft approaches the application of service packs and hotfixes to SQL Azure. One person at my session suggested that they would assume that Microsoft would have done thorough testing of the patch and also that with such limited features available right now that the testing would be able to be streamlined. I tended to agree but decided to point out something I found with their documentation:

Click to see full image

So, they went out of their way to link to four DMVs that they DO NOT support, and the one DMV they do support in this category is not even a hyperlink? Why, exactly, would you hyperlink to things I cannot use? With such a huge usability FAIL, why is it that I would trust them to do thorough testing of any patch when they cannot even handle the usability of their documentation?

I want to love SQL Azure, I really do. I intend to find more and more ways to utilize it whenever possible. But when I see things like above it really makes me question how some of the plumbing is really being handled. I strongly believe that if you take care of the little things then you will inevitably also take care of the big things. And if you can’t do the little things right then you might as well “use the button”.

Use the button, please.

In the past four weeks I have come across a lot of issues with the documentation being borderline unusable. When I was trying to push data to the cloud a few weeks back I talked to Buck Woody (blog | twitter) about a handful of options to which he replied “Why don’t you just use Sync?”

“Well,” I replied, “because I don’t see that mentioned anywhere as an option. Don’t your departments talk to each other? Why is this option not in the documentation?”

I know this is really v1.0 of a product, and that things will get better in time. I truly hope that is the case and we are not going to be subject to having things half-done tossed over the fence at us to use. But the more time I spend with SQL Azure the less comfortable I am that they are taking care of the little things.

And I consider documentation to be more than just a little thing.

The talk tomorrow will start around 1-ish. Our meeting is schedule to go from 12-4PM EDT and I expect that we will start broadcasting around 1PM but don’t get upset if we are a few minutes behind because Rob and I need to spend some time taking care of business and I doubt anyone really wants to watch that. Oh, before I forget, here is the LiveMeeting link for tomorrow:

https://www.livemeeting.com/cc/microsoft/join?id=TJB64W&role=attend&pw=4BMMRD

We have over fifty people registered for the event tomorrow, which would be an incredible number for Rob and I to have for our first event. We are really hoping we can continue to meet here at least once a quarter. Oh, how about a calendar invite? Yeah, that could be helpful. I cannot save .ics files to my current web host but what I can do is configure an event in Google Calendar and share it with you. Just click on the friendly button:

I mentioned that our meeting is scheduled from 12-4PM EDT, and I will start talking at 1PM EDT or so, but don’t expect a three hour talk on SQL Azure. My talk will only last about an hour, including questions. So, if you are planning on watching during your lunch hour, that’s fine, you don’t have to worry about missing the last two hours of the presentation because you won’t miss anything except fifty people eating cold pizza and milling about before heading home.

I hope to see you there, or online, tomorrow!

Anyway, I set about with a few basic questions for myself such as “Self, how many tables are there?” So I did what I normally do whenever I have questions of MS SQL, I took a nose dive into the system information. First up would be to count the objects defined as a user table which I got with this (I was connected to AdventureWorks but feel free to run this against whatever database you are curious about):

SELECT COUNT(*)
FROM sys.objects
WHERE type = ‘U’

Running against AdventureWorks gives me a count of seventy (70) user defined tables. The next question I had for myself was “Self, how many clustered indexes exist for those seventy tables?” This required a little more work. I needed to keep swimming in the system information but now I had to swim in a very particular direction. That direction was towards the sys.indexes view where the index_id was equal to 1. Why a 1? Because that is how Microsoft decided to tag each and every clustered index.

So, the following query would give me a count of the number of tables with a clustered index defined, answering my second question:

SELECT COUNT(*)
FROM sys.indexes
WHERE object_id IN
(SELECT object_id FROM sys.objects WHERE type = ‘U’)
AND index_id = 1

Running this I promptly found that AdventureWorks has sixty-eight (68) tables that have a clustered index.Good news? Not really, no, I was hoping that every table had a clustered index. Why? Great question.

And to top it off, I have no idea which tables out of the 70 are missing a clustered index. All this talking to myself has distracted me from what I really need, which is a script that answers the real question: “Self, which tables are missing a clustered index?” I kept swimming in the system information, doing a loop around the same spot in the ocean I was already at, take the two answers above to arrive at:

SELECT name
FROM sys.objects
WHERE type = ‘U’
AND object_id NOT IN
(SELECT object_id FROM sys.indexes WHERE index_id = 1)

And a-bing, a-bang, a-bung there you go, the results are the following:

ProductProductPhoto
DatabaseLog

What’s that? You want to know why I would even care about this? Well, it turns out that this information is going to be important for anyone that is interested in SQL Azure. Why? That’s a great question. Let me introduce you to my new friend, Error Message 40054:

Msg 40054, Level 16, State 1, Line 3
Tables without a clustered index are not supported in this version of SQL Server. Please create a clustered index and try again.

Yeah, that’s right, tables in SQL Azure *must* have a clustered index defined before you are allowed to insert any data. So, if you are looking to push your database to the cloud, you need to start thinking about your entire design, because there are a lot more factors that need to be considered other than just clustered indexes on each table.

And as luck would have it, I will be touching upon this topic (and others) as part of my talk next week at the Hartford SQL Server User Group meeting. We are looking to have the meeting broadcast live and I will post the event details once I have them.

There was quite a lot of valuable information shared today. Each time I attend events pertaining to Microsoft’s cloud computing services I come away with a deeper understanding of where they are taking us. I think this is partly the result of the fact that more times you hear something the more it becomes familiar and understandable. The other reason is that as the months go by it becomes easier for others to understand what Microsoft is doing as well, including Microsoft themselves. In fact, I would say over half of the audience today were Microsoft employees. I bumped into Rob Walters, for example, as well as a handful of other Microsoft employees looking for a deeper understanding of what services their company is looking to provide.

So what did I learn today? Well, I’m glad you asked. Here are some of the items.

So Many Layers

There are a lot of layers at play here, and a lot of acronyms to go with them. You have software as a service (SaaS), platforms as a service (PaaS, which makes me think of Easter eggs), and infrastructure as a service (IaaS). In there you find Azure (Windows and SQL), Sharepoint, Exchange, Office Communicator, and LiveMeeting. You can be in the cloud, host your own cloud, or have a hybrid model. Microsoft has said that every product they have will be offered as a cloud service eventually, which warrants mentioning here as that is going to dramatically changed the future state of IT over the next 5-7 years.

Time and again the speakers today did a very good job at describing all of these layers and the various ways that we could use existing cloud services to enhance our shops as well as lower overall IT costs. Not just the costs of hardware, mind you, but also lowering the costs associated with development of software. We heard lots of stories about how a traditional project that would take six to nine months could be done in six to eight weeks because you were streamlining the ability to get the right hardware in place in a timely manner.

And anyone that has trouble getting hardware delivered from a vendor can certainly understand the value in that. If cloud services were only about a data center it could be valuable itself, but toss in some software and you really have something to take notice. With platforms and software thrown in you get the ability to quickly change your needs to shifting market trends, not to mention the occasional need for you to handle spikes in usage without needing to purchase expensive hardware.

Point of No Return

The funniest moment of the day was when a question was asked regarding a “reverse-migration”. See, Microsoft will do whatever they can in order to help migrate you to their services. But they really don’t have a lot of interest in helping people migrate off of the cloud. To be fair, I certainly understand why they wouldn’t want to spend much time in helping people migrate away, and that’s fine. The point that was driven home was that once you decide to start using their services it is not easy to decouple yourself from them. So, if you do decide to go into the cloud, you might as well understand that your decision is final.

Think of going to the cloud as if you were having children. Once you have them you can’t put them back.

Corporate Tech Lags Behind Households

One of the sessions was on social computing, which was an (almost) clever way to get me to attend a session on Sharepoint 2010. I love the collaboration tools that they are shoving into Sharepoint, I really do, but I’m not certain that I will get to use them anytime soon. Why is that? Because companies are slow to adopt new technologies.

Remember when going to the office meant that you got to play with “cutting edge” hardware and software? I sure do. At some point in the past fifteen years that has changed. These days, our phones are usually more cutting edge and capable than our desktops at work. And the younger people entering the workforce each year don’t think of social computing as a novelty. No, for them, it is an expected behavior. Facebook account? Expected. Use Twitter? Expected. Have or read a blog? Expected. So for Microsoft to be including those tools in Sharepoint makes sense, but it will be five years before I see them adopted and used in mainstream corporate America.

Ask the (Male) Experts and Get Answers Even If They Are Not an Expert

At the end of the day there was an ‘Ask the Experts’ panel discussion. I looked up from my tiramisu and saw nine men sitting in chairs. That’s right, the first thing I noticed is that there was no woman on this panel. Not sure why, but that stood out quite a bit, especially since there were at least a dozen women from Microsoft in attendance (out of maybe 100 people total), I would like to think one of them had some expertise in cloud computing that they could have been invited onto the panel. Anyway, as the questions went along, someone asked a question on SQL Azure. And then I noticed something else.

Everyone sitting in a chair started turning their heads to look at each other. I went down the line and noticed that there was no SQL expert on the panel! I immediately turned to Rob and said “You’re the only SQL expert in the room, perhaps you should go up and answer the question.” One of the experts started to answer the question and I had a hard time keeping my mouth shut. The question? “How do you connect to, and develop against, SQL Azure?” The answer?

“You simply use the same tools you use right now.”

Um, well, technically that could be considered correct, with a salesman-esque type of leniency, I suppose. When I heard the answer I turned to Rob and said “Wow. He really glossed over a LOT of details with that answer.” I really didn’t want to let this go. My mind was telling my mouth to start asking other SQL Azure questions. Things like:

“Is there a limit to the size of a database?” (Yep.)

“Can I take a backup of my database?” (Nope.)

“You guarantee 99.9% uptime, but will you guarantee me a specified I/O per second?” (Nope.)

“Will you roll back a SQL patch if I need it?” (Probably not.)

And so on. I refrained from asking any questions because I knew I would just be a PITA. Yeah, that’s right, I held my tongue. I know. I can’t believe it either. But I didn’t want to scare anyone and I didn’t want to upset my hosts and I didn’t want to be escorted from the Harvard Club until after they raffled off the xBox.

Overall this was a very good day filled with a lot of valuable information, and I am thankful for the opportunity to attend. I hope to attend 3-4 other events this year that are dedicated to Microsoft’s cloud services, as things tend to change frequently and I want to keep informed on everything Azure has to offer. I do believe that it is where our industry is heading.

*NDA* SQL Azure diagram from MVP Summit

So then, when does it get patched? If you are using SQL Azure, do you get notified in advance that a hotfix or service pack is being applied? Ideally the answer is “yes”, but I’m not certain if that is true.

Let’s say you are an ISV. You have built up a business and are currently using SQL Azure as the back end for what is considered a non-critical application. It is an application, however minimal, that does face outward. That is, external customers use the application periodically. Now let’s suppose that on a Wednesday you get an email from Microsoft that a critical patch needs to be applied to SQL Azure, and it will be deployed on Friday evening.

How do you feel now? Better, or worse?

Where I sit, these are the actions you have available:

Do Nothing

Just sit there and do nothing. Don’t worry about a thing, because you don’t have to. SQL Azure is the new Water Works, it is a utility and it will always be on, and your stuff will always work.

Backup Your Data

Take a backup of your data. Unfortunately, the only way to do that right now is to build a custom SSIS package that will use BCP to pull the data from the cloud and down to your datacenter, where I am certain you have servers available for this anyway. I mean, it’s not like you spun up a business using only SQL Azure in order to save money, right? You did buy hardware, just in case, right?

By the way, unless you are also using Windows Azure, then that SSIS/BCP process is also going to cost you money, because in addition to paying for the SQL Azure database itself, you also pay for the upstream and downstream bandwidth. Can you imagine your cable company charging you for both uploads and downloads, as opposed to a flat rate for the month for a specific Mbps connection speed? Neither can I, which makes me wonder why Microsoft doesn’t offer the same pricing structure, except that I think they want you to use Windows Azure. Either way, you are going to pay money, so you get to pick which way you want to pay. I suppose that’s a good thing.

So, the patch forces you to do this backup, which could cost you money. Will SQL Azure and Microsoft reimburse you for this? After all, if it wasn’t for their patch, then you wouldn’t need to be doing this particular backup (but we know you have backups happening anyway because you are a wise administrator). What if they decide to patch SQL Azure three times a week? Could be a nice revenue stream.

Test

Have Microsoft apply the patch and then you test your application. Wait a minute, this is just the same as ‘do nothing’, except you can have people think you are doing something because you can arrange for testing to happen after the patch is applied. But what happens if the patch breaks your application? Do you think Microsoft will roll back the patch, just for you? I wonder at what threshold of customer dissatisfaction with a patch will Microsoft decide to roll back a patch? My guess is ‘never’, especially if it is a critical patch, but I suppose it is possible.

Still, the thought that my application could break, with little to no warning, causing my customers to be upset, and forcing me to build a patch in a very short amount of time is slightly unsettling. It certainly makes me want to think twice about what can and can not be put into the cloud.

It also makes me wonder if there is a chance that we could get a SQL Azure test sandbox, where Microsoft rolls out patches first and allows for their customers to test changes. Or if it would be possible for me to apply the changes locally in order to test for myself.

I am sure I am being paranoid, but I am a DBA. As Buck Woody once told me “When DBA’s go to a movie we are looking for the fire exits because we always expect disaster.” With SQL Azure still being so new there are a handful of details that need to be hashed out by all parties. I believe patching the cloud to be one of those things that needs to be thoroughly discussed before anyone starts getting too serious about cloud computing.

So, let’s say you are opening up a mine. First things first, do you know what you are mining for? If it were a real mine, say an old-fashioned gold mine, then the answer is yes, you do know what you are looking for. You are looking for all the yellow rocks you can find. Data mining is different in the sense that you have no idea what you are looking for, but you will know it when you find it. Make sense? No?

Think of it this way. You know you have data, you can see rows and columns and numbers and letters. It’s all right there in front of you. But what does it all mean? Well, like a gold mine, you would have all sorts of dirt and rock around you as you tunnel into a mountain or along a stream. With a gold mine you have an idea of what the final product is like, but you are surrounded by tons of material that is fairly useless. Now, let’s say I told you to go into the gold mine and keep digging until you found something unusual. You come back out with a few shiny rocks, some pink, some white, some clear, and some yellow.

Yeah, it kinda feels like this...

Why those rocks? Well, because they were unusual, they stood out from the rest of the rocks you were breaking apart. It was easy to see that something was different about them. Data mining is the exact same thing, you go looking for pieces of data (rock) that stand out from the rest. You may not know what you are looking for, except to say that you are looking for something unusual. Often times people will say they are trying to find hidden patterns in the data, but it could also be the case that finding no pattern can be just as useful. In short, you are looking for those rows and columns that seem to be slightly different than the rest.

So, your mine is open for business, you are excavating tons of rocks, and a bunch of them are yellow. But how do you know they are gold? Well, someone needs to do some analysis on those rocks to verify what they are. So, if you are mining for gold, the steps would be as follows:

1. Go digging through dirt and rocks.
2. Pull out the ones that look different.
3. Tell someone which pile of rocks is the one you think contains gold.
4. Have someone analyze the rocks in the pile to confirm.
5. Profit.

With data mining the same principles apply.

1. Go digging through your data.
2. Pull out the data that looks different.
3. Propose a hypothesis about why the data is different.
4. Perform some analysis on a larger set of data to see if your hypothesis is valid.
5. Profit.

Now, I am simplifying things here greatly. I am not talking about confidence intervals, or different statistical models, or anything in depth. I am just trying to explain that there is a difference in the mining of data versus the analysis of data. To me, at least, the analysis phase comes later, after you have formed a hypothesis, which is brought about by some mining.

I have seen some people describe data mining as a particular data analysis technique. I suppose that is true as well. For example, you wouldn’t start mining for gold unless you already had a hypothesis that there was gold in ‘dem ‘thar hills. That is also a fair definition, but it does highlight that the mining is still something not entirely equal to the analysis.

They are different tools, and different methods, but with a common purpose. In the case of Business Intelligence it is to get the right information, to the right people, at the right time, so they can make the right decisions.

Data Mining Tools

Microsoft has two different flavors of data mining tools that you can add easily into your office products, specifically for use with Excel.

If you have access to an instance of SSAS at all times, then go ahead and download the tools found here:

http://www.microsoft.com/downloads/details.aspx?FamilyID=7C76E8DF-8674-4C3B-A99B-55B17F3C4C51&displaylang=en

And for those times when you are not connected to an actual SSAS instance, but are connected to the intertubz, then you will want to download this set of tools:

http://www.sqlserverdatamining.com/cloud/

Feel free to load both of them if you want, they can be run side by side. Why would you want to do this? That’s a great question. Let me explain by continuing the gold mine analogy.

So, if you were serious about going to mine for gold, would you grab a pick-axe and make way for the Yukon Trail? Probably not. And even for those people that did do exactly that during the various Gold Rush eras in history, they only did so because someone else had already found some gold. In other words, the preliminary exploration had been done, gold was discovered, word spread, and before you know it we have enough Yukon Jack to last a lifetime.

If you were going to open a gold mine today you would look to do some preliminary exploration as well before you ever broke ground somewhere. Think of the tools that you download as exactly that: tools that help you (or others) do some preliminary exploration. In this case, exploration of data, no dirt. The tools help to highlight chunks of data that you may find to be of some interest.

So, the tools help you with some preliminary exploration, at which point you can really start to mine for more information, then you form a hypothesis, perform some analysis, and a-bing, a-bang, a-bung you are on your way!

This past weekend I started thinking about what it really means to have your databases hosted in the cloud. My first thought was that it struck me as being exactly the same as an Application Service Provider. I mean, MS SQL is an application, right? And you would be looking to have someone outside your company provide you that application as a service you would pay for. So…is there a difference between the Cloud and a good old-fashioned ASP model?

Here is the wiki entry for ASP. Decide for yourself. Note the mention of how ASP is also known as SaaS, or “Software as a Service”. Perhaps putting your database in the Cloud would now mean we have a “Database as a Service”, and that is the difference, that you are focusing on a specific piece of the software/application being offered.

Now, while we argue over the definitions of all of this, and where to draw the lines, and compare and contrast architectures, let me offer you this article, which asks the simple question “Is Software as a Service Really Cheaper?”

http://software.silicon.com/applications/0,39024653,39397645,00.htm

Ok, quick show of hands. Who thinks Cloud Computing is going to take off faster than an F-15? Now, from that set of people who still say “Hell-to-the-yeah”, does that mean your business is doing so because it will be a way to reduce costs? If so, have they ever tried to define all of the costs associated with a traditional ASP/SaaS/DaaS model?

The Cloud Is Not New

The idea of pushing your data to somewhere else is not new. It has been around for decades. In the past ten years some different business models have sprung up. One of them was called ASP, which still exists today. Some ASP’s have changed their identity to something more hip, calling themselves SaaS, but they are essentially the same. Now, how many people are putting their heads in the Cloud because they believe it is something shiny and new, but have no idea if it is actually something better?

I usually ask myself all the time “am we in a better place now than a year ago?” The answer is almost always yes. On the surface it would seem the Cloud is indeed the future, but I would be very inclined to take a second and third look at all of the hidden costs associated with a Cloud model. If the Cloud really is an old idea being spun differently, then there is no reason to believe that there is going to be an incredible push for the Cloud very quickly. At least not by business that do some reasonable research into the costs associated down the road with one form of architecture. If your business changes ideologies and architectures frequently and doesn’t mind spending money freely, then you will probably adapt to the Cloud quickly.

It will take some time for business that do adapt to the Cloud quickly to publish some metrics regarding their costs savings before everyone else will follow suit. Believe me, business will always look to save money first. If the Cloud does save money for them, then other business will make plans to start shifting their architecture to the Cloud as well, which could mean more than a handful of DBA’s will need to package their old selves as something shiny and new as well.

I still believe we have more than a few years before we have to worry about all of this. Sure, change is coming, but not as fast as some would like to believe, regardless of all the NDA gossip going around the rumor mill these days.