Friday, March 16, 2012

On Reviews...

Robb explains his review methodology in response to a critique of reviews from Volquartsen Vuurwapen. As someone that's done a bit of reviewing (and, admittedly, probably a little bit of what Vuurwapen complains about), I wanted to add my 83¢ (that's 2¢ adjusted for Obamaflation).

There's a bit of tempering, for me - Robb rightly points out the danger of falling into the old gun rag trap of every gun being the brightest and shiniest and greatest thing since the percussion cap out of fear that a "bad" review will cut off the T&E stream. This, though, is balanced by the shame I would feel if someone went out and bought something based in part on my recommendation and I didn't fully cover the item, faults and all.

And let's face it, there's nothing out there that's perfect.

I've reviewed a fair number of new firearms for T&E purposes, and I have yet to have one that didn't have some minor glitch somewhere along the way. It might be a light strike on certain ammunition, or failure to load a round with a bulge near the crimp, or an occasional stovepipe. None of the pistols I've reviewed has run flawlessly. None. I've taken notes every time I've brought a T&E gun to the range (well, most times; sometimes a T&E gun is in the range bag and will get shot for fun). Every gun has a glitch or two along the way.

And as long as I include these glitches in my reviews, I feel that I've done my job as a reviewer. I won't review a gun without putting at least 500 rounds through it, and generally strive for closer to 1,000 rounds to be on the safe side. Part of this testing is letting the gun get filthy dirty and seeing how long it takes for something to go wrong (and no, I'm not including failures from a dirty gun in the last paragraph about glitches). While I will admit this is far from a thorough or exhaustive procedure, I'm confident that 7-10 range trips of 100 rounds or more each will give a pretty good snapshot of a firearm's performance.

Case in point: DaddyBear bought a Smith & Wesson 22A based in part on a review I did for Guns, Holsters, & Gear. Now, in that case, I was pleased as punch to hear that, because the 22A is just a fantastic little rimfire at a great price, and the model that I was shooting was nothing short of amazing. It got finicky and dirty at the end of each range session, but until it got to the point where it started bogging due to grime, it was a sheer joy to shoot. For the money I don't think there's a better gun out there, and I'll defend that suggestion all day long. Even feeling that way, I was apprehensive as hell until he posted that he was happy with it - I'd have been crushed if he hadn't been.

The process, really, is moderately self-selecting. I've had offers of items to review that I've passed on, sometimes because I didn't know enough about the product to properly test it out, sometimes because I didn't have enough time, or, in a few cases, I took one look at the item and couldn't possibly see how it could work in a real life setting. The items that I have reviewed have been things that have interested me to begin with - my first reviews ever were for the LaserLyte RSLs, which caught my eye in a gun magazine and I posted about them.

There won't be a Cobra Arms Derringer showing up any time soon, in other words.

I will typically post three different times about a review article. There will be a "hey, look what I got" post; a post with initial impressions, and then a "down the road" post with information on how the item held up over the long(er) term. I've got LA Police Gear Operator tactical pants that I received several years ago that have made their way into general rotation and literally get worn on a weekly basis. The SIG SAUER .22LR conversion kit for the P226 that I got last year has taken up permanent residence on the P226. Generally, I'll try to get some honest use out of whatever it is I'm trying out.

Lastly, I'm not a grand master of gun-fu. I'm not an IDPA or IPSC participant (although I would like to be). I don't put thousands of rounds downrange every weekend; heck, I'm lucky if I can get to the range more than twice a month. I'm just an average guy who happens to like guns. If you're looking for a review of the latest Blastomatic 5000 and how it holds up under extreme torture tests, it's unlikely to be here. If you're a bullseye shooter looking for thoughts on the latest custom built match gun, also not likely.

If you're a regular gunnie looking for a regular gunnie's thoughts on guns and gear that regular gunnies use, I'd like to think that MArooned is the place.

That is all.


Dave H said...

I like your reviews, Jay. When I'm interested in a product I want to know what people -think- about it. I can find specs for pretty much anything on the 'net, and if I want test results I can usually find them in a print magazine or the Box 'o Truth.

But specs don't tell the whole story. A pistol with a 1.2 pound trigger pull might be great for target shooting, but if the trigger guard is too small it might lead to an accidental discharge. That's not something you get from a spec sheet. You learn that from somebody who took it to the range and had a couple of ADs before he decided to put it away.

To me, blog reviews are like talking to a friend to see what he thinks. It's informal, unscientific, and very subjective, but if the blogger uses the product the way I would use it then it's a fairly safe bet that if he's happy with it, I will be too.

That's why militaries run field trials of new weapon candidates. Specs and costs may make a weapon look like a miracle, but it's a waste of money, ammo, and lives if the troops can't make it work.

Pakkinpoppa said...

I'll admit, I'm pondering that Crimson Trace Lightguard for the 26L that'll get toted as the decline gets a little steeper. Was it based on your review? Yes, as the first place I'd heard of it was here.

The drawback of said piece is...being, um, self customized (I did say "26L", and I did it myself), I will have to make sure the item fits aboard prior to purchase.

I also admit I'm falling into your "gun nut" category, as a second safe is necessary before obtaining (m)any more blasters.

DaddyBear said...

The 22A is a great example of why I pay a lot of attention to reviews done by other bloggers. If someone I know is in a lot of ways similar to me likes it and can point out the good and bad about the gun or piece of gear, then I know that I can probably expect to have a very close experience.

You also point out another reason I shy away from T&E on guns: ammunition. I have a budget for ammunition, and having to budget several hundred dollars for ammunition for a gun that I will probably send back to the manufacturer at the cost of shooting my EDC and hunting guns with that ammunition would hurt. Once I have more experience, time, and ammo budget, maybe I'll re-consider.

Jay G said...

No kidding. I've had three .45 ACP guns recently - talk about straining the ol' ammo budget...

Obviously I need to T&E more .22LR... :)

Thanks DB.

Anonymous said...

I am very fortunate in my life to have known Elmer Keith (friend of my grandparents) in my lifetime.

That said, I have enjoyed your reviews much more than many others I have read in print mags or online.


Ed said...

A "failure to load a round with a bulge near the crimp' may have just saved you from a "ka-BOOM!!!" when the case ruptures.

Did the bulge cause the round to exceed SAAMI spec?

Do you expect that the manufacturer open up the chamber diameter to handle out-of-spec ammunition, reducing velocity and possibly causing leakage of propellant gas back towards the breach and you, the shooter?

If anything, your criticism should be towards the manufacturer (or reloader) that produced that round with the bulge.