THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | Join | Help
in Search

Aaron Bertrand

Aaron is a senior consultant for SQL Sentry, Inc., makers of performance monitoring and event management software for SQL Server, Analysis Services, and Windows. He has been blogging here at sqlblog.com since 2006, focusing on manageability, performance, and new features; has been a Microsoft MVP since 1997; tweets as @AaronBertrand; and speaks frequently at user group meetings and SQL Saturday events.

Digesting the bitter pill : feedback from SQL Bits

I gave two Denali presentations at SQL Bits (you can download the deck and samples here). I'm a little disappointed in the feedback, to be quite honest. Don't get me wrong, I got some really positive and appreciative feedback, almost all 7s and above, and lots of 9s scattered about. A few examples:

"Good choice of subject matter - filled in all the new bits about Denali that had not been presented in other sessions"

"The only session I attended where the presenter told us where the resources could be found. And he had already put them there! Very much appreciated. I was able to spend the time listening instead of writing."

"Good content, interesting subject matter, Aaron did a great job with good, relevant demos. He answered questions well and was very assured. Good stuff."

Here are my average scores compared to the conference top & average (click to embiggen):

 

So, I was in line with the average for the entire conference, and not really that far off the top, all things considered. But among the comments, it quickly goes downhill. There are always a few, "well, can't please everyone" instances, and I completely understand that. But the one I loved was this one:

"BI is a major component to denali and to skip BI was a great let down, I think the session should have been called 'what new SQL features in Denali (Nothing on BI)' so I could have skipped the session."

Let's review. The abstract for this session starts off with:

"This presentation will cover all of the new engine and tools features coming our way in the next major version of SQL Server."

And go ahead, look at the slide deck. The very first content slide says:

Overview

  • Changes to setup, engine, and tools
  • Will leave BI stuff to BI pros
In addition to my very clear verbal sentiments at the beginning of the session that I would not have time to go over BI features, what else could I do to make this more clear? Don't blame others because you sat through the entire session waiting for something that obviously wasn't coming. I also didn't talk about migrating to Denali from Oracle, should I also include '(Nothing on Oracle)' in the session title? Where do I draw the line? I'm sorry the abstract and my repeated warnings about BI were not enough for you (in fact I even believe I said, "if you are here for BI, this is not the session you want to be in"). I can't do much more than that.

And for the two people who gave me 3s and 4s (out of 10), with absolutely no comments, I've made a mental note of your names, and you better wow me when you present and I'm in the room.

Thanks to Allan, Simon et. al. for compiling the feedback and making it 'pretty' - well, as pretty as you could possibly make it, anyway.

 

Published Monday, May 16, 2011 3:08 PM by AaronBertrand

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Jen Stirrup said:

Hello Aaron,

I didn't attend your sessions because our timings clashed, unfortunately. I was one of the other speakers.

For me, I think that the takeaway points is that your positive feedback was very positive, and has the hallmarks of an mature speaker.

From your abstract, it's clear to me that you were specific about the direction you were taking. It sounds to me as if you have done all you can to align people's expectations to your session. It sounds unfortunate to me that, for some unknown reason, some individuals didn't pick up that the session had a different direction than being oriented specifically towards business intelligence.

The positive comments are worth noting! I'm sure that you'll see there are different positive messages given to you by those who benefitted from your session. It sounds to me as if you did well and I'll look forward to being able to attending one of your sessions in the future.

Kind Regards,

Jen

May 16, 2011 2:44 PM
 

James Berry said:

I didn't catch your session at SqlBits, but I was in the audience at the Kent UG meeting where you presented from the US over video conference - and thought it was a very well balanced & informative session and I enjoyed your presentation style a lot.

Perhaps you should take the median rather than the average ;-)

Best wishes

James

May 16, 2011 2:47 PM
 

AaronBertrand said:

Thanks so much for the kind words, Jen. And thanks also for confirming that there is not much else I can do to set proper expectations (other than trying to squeeze some BI content into the deck - I almost always go over, so not much chance in that).

May 16, 2011 2:47 PM
 

AaronBertrand said:

Thanks James.

May 16, 2011 3:00 PM
 

jonmcrawford said:

Think you might need to take that "Trying to be objective" tag off the post, because you clearly are not. Frankly, I'm surprised by the tone of this post. You knew you mentioned that BI wasn't in it, so why did you need to spend 50% of your commentary on that person? They were misguided, let it go.

As for the rest, your take is (paraphrased) "Thanks for the good marks, and the rest of you suck. I'll be giving you bad marks out of vengeance when you next present"?

I've been reading your blog for a while now, and know you're better than that.

May 16, 2011 3:02 PM
 

Peter Schott said:

Aaron, maybe your next presentation should include one of those densely packed words outlining everything you won't cover. Make it ridiculous, use Wordle, whatever. If you said you wouldn't cover something on the 1st major slide, that's a silly comment. Haven't seen your presentation, but hope it becomes available online soon.

May 16, 2011 3:05 PM
 

AaronBertrand said:

Jon, "out of vengeance"? I don't think so. If someone has the gall to give someone a 3 out of 10, and not explain why, they better put on a good show when they're on stage. I don't think that's an unfair expectation at all.

May 16, 2011 3:11 PM
 

jonmcrawford said:

Ah, incomplete information. You should have mentioned that they did not tell you why in your post. As written, it sounded like you were out to get the people who gave you bad marks, simply because they were bad marks. I stand corrected, and relieved. ;)

May 16, 2011 3:47 PM
 

Ben Thul said:

The problem with statements that contain "...had better..." is that there's an implied "...or else".  The implied consequence in this case is that you're itching to give them bad marks on their next presentation.  There's a Chinese proverb "He who seeks vengeance must dig two graves.".  I agree with Jon.  The question is: are you receptive enough to listen to the criticism and just let it go?

May 16, 2011 3:51 PM
 

AaronBertrand said:

Ben,

Listen to what criticism? They gave a 3 and no comments. I'm not sure what your feedback etiquette is, but to me that is just rude. Common courtesy dictates that if you tell me I did a terrible job, give me some hint as to why, so I can improve for other attendees in the future.

What I'm suggesting is that if I'm in their session it is going to be tough to give great marks when I know what high expectations they have. Does their behavior not guide me to judge them the way they clearly judge others? If it turns out they are not as great as they expect others to be, I might even have the courtesy to tell them WHY they disappointed me so fabulously.

May 16, 2011 3:54 PM
 

AaronBertrand said:

Sorry Jon, I have updated the post with some emphasis on why I think that behavior is unacceptable. If you give me bad marks and tell me why, I'll do nothing but thank you, I promise. If you give me bad marks but don't have the courtesy to tell me why...

May 16, 2011 3:59 PM
 

Ben Thul said:

What criticism?  The criticism that at least two people now have said that your statements here imply premeditated revenge.  I'm not saying anything about how you should take the initial speaker reviews, only that what you've said here doesn't do anything to win friends and influence people in a positive way.  Of course, I may be in the minority here in which case you're free to ignore me.  My feelings won't be hurt any.

May 16, 2011 4:13 PM
 

AaronBertrand said:

Ben,

I'll be quite happy to leave them positive reviews if I ever attend their sessions and they are excellent. This is not about vengeance or revenge at all - simply a warning that they will be judged to very high standards.

As others have pointed out both here and on twitter, the type of person it takes to leave a drive-by 3, and not explain why, is very unlikely to be a presenter him/herself.

That doesn't mean I should not be allowed to try to motivate them to think a little harder about their reviewing process in the meantime. Leaving a 3 with no comments (and therefore absolutely no chance for me to learn from their "criticism") is just inconsiderate. I'm sorry if you don't agree (you'll notice that Jon updated his comments once he had the full story).

May 16, 2011 4:19 PM
 

Matt Whitfield said:

I feel your pain, sir. I had pretty much exactly the same - apart from the fact that I didn't have huge numbers of people (last thing on a sunny Friday next to the beach at the same time as some awesome speakers) - so it really hurt my ratings.

Interestingly enough, the abstract in the book for my session was different to the one one the site / posters - so I think I messed up when submitting my session.

But yeah, I don't really feel like there is much I can do with the 'no name no feedback but have a 3' kind of people...

May 16, 2011 5:54 PM
 

RichB said:

So, you consider a 'drive by 3' to be rude?

Seems odd to ask for a metric, then feel narked that the metric is filled with a value you don't want.

Sure, it's nice to get constructive feedback, but this attitude just stinks to me. Can't criticise if you can't suggest a better way to do it?

Might as well say you can't vote if you aren't a paid up party member.

May 17, 2011 6:31 AM
 

AaronBertrand said:

RichB, let me repeat, once again. It is not the 3 that I am narked about, it's that the 3 was not explained in any way whatsoever. This just seems like basic, common courtesy to me... but perhaps non-presenters are oblivious.

Let's spin this a bit. In your review with your employer, would you be happy if your boss told you, "Hey Rich, you've done a terrible job over the past year. Sorry, no raise and no bonus." That's it. No context, no helpful information about why you did so poorly. Doesn't that leave you wondering what you did wrong, and how you can do better this year? Shall I suggest that you asked for the review and should be happy with whatever information you got, even if it is negative, useless, and not actionable?

May 17, 2011 7:45 AM
 

David Wimbush said:

I imagine that, while it must be great to get mostly good ratings, it's the comments (positive or critical) that you really want as a speaker. What did I do well? What did I do poorly? The comments are gold dust. Otherwise it's like trying to tune a query without being able to see the execution plan.

At least you got to top up your tan under those lights. They were fairly bright.

May 17, 2011 8:31 AM
 

Aaron Bertrand said:

I received a lot of grief at SQLRally when it was revealed that I have never submitted to speak at the

May 17, 2011 9:03 AM
 

Richard Douglas - @SQLRich said:

I enjoyed your session at SQLBits very much, especially the quip about Microsoft changing the HADRON terminology because of typo's :-)

It was great to have an insight into what is coming up next version of SQL. One of the problems may be that SQLBits has gone from paperbased feedback forms that you used to fill in during the session to online which people will fill in when they arrive home.

It can make it difficult to remember exactly why you are critiquing unless there was something that you really disagreed with. Most people will just go by the feeling they remember rather than specifics.

Rich

p.s. I didn't give you a 3

May 17, 2011 9:19 AM
 

RichB said:

Repeat all you want, I did not miss your point at all, but you bypassed mine - fair enough it's your blog spot.  

If it is such a drama to receive metricised but uncommented criticism, why add a metric at all?  I guess all the good marks were water off a ducks back.

There are a lot of people who do not feel comfortable, or willing to put the time into, expressing written feedback, many who fear the confrontation even in an anonymous poll.  They are usually much more comfortable with leaving a grade rather than expressing an opinion.  I guess this entire post explains why.  Would you rather silence?

I'm guessing the fact that you've made a note of who indicates they are on 'the circuit' too, possibly friends or colleagues, in which case the tone of this post is able to be read one way.  On the other hand, if these are effectively strangers, it is a pretty unpleasant sentiment to express to the world on the internet, especially from someone who usually has interesting things to say.

May 17, 2011 10:12 AM
 

AaronBertrand said:

RichB,

I'm not sure which point of yours I bypassed. Let's address them individually and maybe you can tell me what I've missed.

First, you asked if I thought a "drive by 3" was rude. I think it's obvious that I do - the whole point of feedback is to learn something from it, just as the whole point of me presenting in front of a room full of strangers is so that you can learn something. So I'm guessing this question was rhetorical.

The next point you made was that I didn't like the 3. I addressed that, by reminding you that it wasn't the 3 I didn't like - I'll take 3s all day, if you tell me why. Then at least I can do something about it and not sorely disappoint some audience member in the future for the same reason. Since I have no idea what that reason is, the 3 is utterly useless to everyone.

You say that my attitude stinks. Okay, that's your opinion, and to you and a few others, you're right. How am I supposed to respond? No it doesn't, neener neener? I have my opinion, just as you have yours. You can tell me my attitude stinks all day, but that doesn't change my opinion.

You ask if you can't criticize if you can't suggest a better way to do it. Yes, I agree there is some truth to that. If you walk around all day telling your co-workers, "You suck." "You suck too." "Oh, you suck even more." Don't be too surprised when one of them punches you in the face. However a statement like "it sucks when you do x" or "I hate it when you don't do x" can at least move you toward some kind of correction or resolution.

Your last point was about politics. I'm not touching that one for reasons that should be obvious.

I noticed that you didn't acknowledge my counterpoint about your employment review. Do you think it's completely irrelevant, or that maybe there is some parallelism there? What would your reaction be if your employer did that to you? Do you think it's okay for them to just tell you you suck, because they don't feel comfortable with expressing their feedback?

Again, my expectations here aren't that high (if you're going to tell me I suck, tell me why, so I can do something about it), and I've already agreed both here and on twitter that my reaction might be perceived by some as a little harsh. That doesn't mean my expectations are unrealistic or that I shouldn't be disappointed when this kind of thing happens.

May 17, 2011 10:36 AM
 

Neil Hambly said:

Aaron

I attended your session and I found it to a done to a high standard of presenting (Packed room mostly) & in-line with the abstract and slides, I was also a fellow speaker @ the last SQLBits I have received my feedback (practically a mirror or yours) and had one feedback with low scores all 4,5's but no name or comments..

Frankly all I can gain from that is they were not impressed for one reason or another..not much else I can do with it and frankly I don't expect to always get high scores across the board and THE MOST USEFUL feedback I find is actually the comments not the score itself

Finally I don't think anyone is probably more critical of my session than myself

I think we just need to ask for better feedback from our audiences, if taking a few seconds to explain why that is of benefit to us and them in return

May 17, 2011 11:39 AM
 

On Feedback… And a Plea To Event Organizers | SQL Server Blog by Mike Walsh said:

May 17, 2011 11:52 AM
 

John Newman said:

Here's some constructive feedback: Why are you sharing your feedback with the world? Just so that you can have a nice pity party now, then brag when or if you do better? Feedback is something for you to learn from, not something to publicly air. That's why it's given to you privately by conference organisers.

May 17, 2011 2:05 PM
 

AaronBertrand said:

See John, now isn't that much more useful feedback than something akin to, "your blog post sucks?" I think the people giving feedback can learn just as much from something like this as the people providing the sessions in the first place. As far as I know, organizers give us our feedback in private because we may not necessarily want to share our feedback, not to prevent us from doing so. I'm not looking for a pity party or hoping to brag later. Just wanted to shed a little light on how people providing feedback might be doing it wrong (and I'm not the only one to do so).

May 17, 2011 2:55 PM
 

RichB said:

Honestly I'd rather receive the 3 than not know.  Sure some explanation would be good, but it isn't necessary, a bit of self inspection based on the fact that someone feels that way should give one an opportunity to improve if their ego and self belief allows it.  It seems genuinely strange to me that someone would rather not know.

Your conitinued aggressively defensive attitude fully backs my reading of it, and Bens, Johns and RichDs points fits in quite well with my concerns.

If you prefer silence to a poor metric, then with luck that is what you will get from anyone who has read this thread, as I truly doubt that it will lever more complex feedback out of those who don't like to leave it for whatever reason.  

But please, do let us know if it was left by someone on your circuit - a friend/colleague/competitor or just a visitor.  It colours the whole thing, which has clearly aroused interest in several commentors :)

R

May 18, 2011 6:30 AM
 

A different Rich who's surname is also B said:

I too thought it was a little odd to get so grumpy about the people who gave negative feedback without a comment. I bet you weren't worried about people who gave you a high mark who didn't explain why.

People are entitled to put what they want on a feedback form- it may frustrating not to know what you could have done better, but it doesn't deserve all that wrath and anger, as if they've done it to bring your marks down.

May 18, 2011 8:01 AM
 

Frank Hillshire said:

Where do you Riches see "all that wrath and anger"? I don't see any of that, I just see disappointment, and I think you folks are over-reacting a bit here. Just like a couple of folks are free to put whatever unhelpful information they want on a feedback form, so is Aaron allowed to express his opinion about that information. Which I took as, come on guys, put yourself in the presenter's shoes: would /you/ like to get a 3/10 with no comment?

May 18, 2011 8:28 AM
 

A different Rich who's surname is also B said:

Fair enough to express an opinion "I've made a mental note of your names, and you better wow me when you present and I'm in the room" is a bit objectionable.

May 18, 2011 8:44 AM
 

RichB said:

It's the internet, and someone was wrong... ;)

May 18, 2011 10:56 AM
 

Frank Hillshire said:

A different Rich, maybe you find that objectionable, the same way Aaron found the low rating with no comment objectionable. Sometimes not saying something can come across as lazy or offensive the same way saying something can. And maybe you're reading too much into it anyway. The way I interpreted it: people who live in glass houses...

May 18, 2011 11:03 AM
 

KarlL said:

Here's the thing - the fact that there's even a hint of retribution in Aaron's comments means to me that he's not mature and introspective enough to be dealing with this kind of feedback.

Getting a bit discouraged by a low score without feedback is fine.  Using a blog to publicly air that discouragement, with a promise that you'll harshly judge the people providing it is just childish, and highly unprofessional.

One thing I have definitely learned from this thread: If I do attend a session of yours in the future, I'll be damned sure not to rate you one way or another unless it's anonymous.

May 18, 2011 1:13 PM
 

AaronBertrand said:

Well I'm kind of sad that my comment had a different effect than intended. My point was merely to try and encourage folks to be honest about their feedback - if I did terrible, that's fine, and I'm certainly mature and introspective enough to deal with that kind of feedback. But it would be much more useful if you tell me I did terrible but also told me WHY I did terrible. What is the point of telling me I did a terrible job, but not bothering to mention anything that can help me improve? Are your standards so high that you can tell someone they earned a 3/10 from you, but not tell them why? That was all that I was intending - was to think about the standards by which you will be judged when you're the one in front of the room.

For the record, I'm not going to change the way I rate other presenters. I'm going to continue rating them fairly, and I'm going to continue including comments when I think they will be constructive. Does anybody need to know why they got a 9 or 10? Doubtful. Pat-on-the-back comments like "good job" are implied at the high end. Context is much more useful when it is a 3 or a 4. A single comment that can help that become a 5 or a 6 next time is going to outweigh a hundred 9s or 10s, every time.

May 18, 2011 1:31 PM
 

dan said:

Hi Aaron

I would ignore the low marks with no comments to be honest - it really doesn't matter how much information you give to the audience, you will still end up with people who think that a talk doesn't give the right content.

Also, sometimes the people who complain about a particular topic being missed out weren't in their seats for the slide that says "No BI".  maybe repeat again after ten minutes (or watermark your presentation =)  ).

Really enjoyed your Bad Habits talk and catching up with you at the stand, honestly Sharepoint database monitoring would be a great idea !!!

cheers

October 3, 2011 4:50 AM
 

AaronBertrand said:

Thanks dan, looking back I must admit that I was a little over-passionate about this issue. I have no problems with receiving poor marks for my presentations, I just feel like if you can tell me I sucked, you can tell me why I sucked. That's the only way I'm going to get better.

October 3, 2011 9:06 AM
 

Aaron wimbush | Trusteesalesol said:

October 27, 2011 11:21 AM

Leave a Comment

(required) 
(optional)
(required) 
Submit

About AaronBertrand

...about me...

This Blog

Syndication

Powered by Community Server (Commercial Edition), by Telligent Systems
  Privacy Statement