Question on reviewing / suggestion

[Andrew]

Good one William

With the disclaimer: Don't take that comment seriously!  William! 

[skinnyfat]

With the disclaimer: Don't take that comment seriously!  William! 

too late LOOL.

It is refreshing to have some honesty

[Timber_MG]

*just to move back the post form the split off thread*

I think a Stereophile'esque measurement section would be great (cause manufacturers are very inconsistent, try Spendor sensitivity as an example) but with iso-metric representations of polar data it would be great to allow users to choose suitable speakers to their situation and to see more than just a static single FR plot or something of the like. What would others here see as sensible measures to include in reviews?

This would also put loudspeakers from the likes of Vivid or others into a proper light against some of the lesser engineered alternatives for example but also show up good value items in review, the kind that might get lost elsewhere.

[Shonver]

This would be useful to technically-minded readers. But I think also it is wishful thinking in that it would require extra resources.

What consumers really need is reviews to be done in a context that they can relate to. One step toward this would be to list other products in the same price bracket, to "also try".

[joel]

As a starting point, I know what my system sounds like and will only review one component at a time.

This means that any differences I hear, must come from the latest piece of equipment installed.

As far as possible I will level match components to ensure that the differences heard arent caused by subtle level shifts, and will also perform A/B comparisons using the same source material.

I use two analogue tools that i find are extremely good at helping me analyse equipment, with these being my feet.

if they are tapping along to the music and there is a smile on my face, then the review product will probably get a good review. If they aren't moving then no matter how technically proficient a product may be, it is failing to convey the essence of the music I'm listeng to. Such a product probably wouldnt score too well in my reviewing opinion.

My firm belief is that music should come first and brand names and specifications a distant second.

On to objective measurements.

I have Linear X's LMS in my possesion, and can perform semi anechoic measurements, but it takes time to set the system up and its not viable to test one speaker at a time. This means that in order to run a meaningfull test session, we would have to hold on to review speakers for a lot longer than most distributors would be happy with.

Often we have speaker for as little as a few days and there is barely enough time to listen to them, let alone hold on to them untill we're ready to perform tests on them.

I hope this helps

Regards

Joel Kopping
 

[williamkelly]

Sure thing. I speak only for me - but I think that blind listening tests ensure objectivity. Take a R2k CD player and listen to it really hard, and then compare it with a R20k CD player - without knowing which is which. Think you can tell which one is better? You may be able to tell that there is a difference, if you're good and have fresher ears than mine - but as to which one is better - well, now, that's a much harder question to answer. That's the point. However I wouldn't expect anyone to review on blind listening tests simply because we are subjective creatures and 25% of the kicks I get from my hi-fi are from my prejudices and subjectivities... (!)

[skinnyfat]

if they are tapping along to the music and there is a smile on my face, then the review product will probably get a good review. If they aren't moving then no matter how technically proficient a product may be, it is failing to convey the essence of the music I'm listeng to. Such a product probably wouldnt score too well in my reviewing opinion.
 

Welcom Joel

It warms the cockles of my heart to hear this . Too many people get caught up in numbers, stats and urban legend and forget that its the experience that counts (and dare I say cleaning LP's with a lint free swab doesn't qualify as an experience )

If a reviewer is known to have this approach then I feel readers of his/her reviews would be prepared to make better auditioning choices based on these reviews.

[jamster]

I recon the first step is to objectively assure one self that there is in fact a difference between devices when factors such as gain, FR tapers in the range 20-16 (let's leave some room for brickwall filters or the "other" kind) have been accounted for. One so often sees especially salespeople trying to address problems in the FR/acoustical domain with what amounts to a placebo remedy seeing that suggested "upgrades"/replacement equipment is technically close enough to be wholly inconsequential.

In the acoustical sphere a pair of loudpseakers at a given cost point can often outperform a "higher-end" unit in less well set-up acoustics and electronics can only do so much to remedy the situation (and most of that is usually shunned by the "purists"). Often the "higher-end" results more in marketing engineering than acoustic engineering aimed at providing value (though there are notable exceptions) and unfortunately device reviews of a subjective nature are some times pulled in to aid and abett imo fraudulent sales practice by incompetent salespeople (and I find that boutiques run rife with this breed)

An example: If a loudspeaker with a large flare in the HF power response (say at 3-5kHz like a well reviewed for on axis FR brand that is widely regarded amongst consumers) is integrated into a live acoustical environment, no CD player, amplifier or cable will do anything to address the resulting upper midrange prominence (and EQ will only leave you with a duller reverberant field and a hole on axis).

This is also where I see the role of AV magazines in educating the public at large but also giving them sufficent objective information by which to make informed and relevant choices. So often the choice of a loudspeaker is made in the (usually damped) listening room with little or ill-considdered (due to technical incompetence often) information or lacking knowledge as to what actually matters in the specific acoustic situation of the client. I get the impression that this lack of knowledge often sees individuals looking at items in the "signal chain" as the source of their problems where the acoustical practice might show to a very different set of problems in reality (commonly encountered in any domestic situation)

</rant>

*edit: grammar*

I'm struggling to understand this post. Do you believe this nonsense?

You say that manufacturers and fraudulent, except when they aren't. And that people should listen before they buy, becuase a system that sounds bad doesn't sound good?

You also say that even if manufacturers say it sounds good, it may not sound good.

Finally, AV magazines should tell people to only buy stuff that sounds good.



Top marks for this big words, though. I like the word 'obfuscate'.

[jamster]

When reviewing I know my system quite well and so have a basis to contextualise changes to it. I try to only change the system to accommodate only the review component so that I revert to the mean. And then I try and listen when I am in a jolly jovial good mood which is something equipment suppliers don't understand -  a big fat bribe puts me in the most jovial and jolly good mood you can imagine. Yes! I can be bought and relatively cheaply too.

And then its up to the component to convince me. Sure, the process is biased and unfair - and that's just the way it should be. You don't see Clarkson liking BMW simply because its BMW do you? What's to like after all? The man is perfectly correct in his bias .

But overall on the balance of all reviews some components will stand higher than others. This might be an indication of superiority - the one thing I have learned over the years is that there is no way to really tell what works for other people. So these days I don't really bother about it - I report on what works for me whilst trying to accentuate the positive aspects of the component. Occasionally there are none and I will send the component back un-reviewed - fortunately that hasn't happened in ages now.

Re matching components - hmm. I used to subscribe to that philosophy when I was young and foolish. It's a bunch of baloney. Do a blind listening test on CD players ranging from R2k to R22k and you'll see what I mean. Worse, engage in the same test with the vinyl crowd and the point should be crystal clear... Whatever works for you is what works - and precisely the same applies to reviewers, along with our prejudices, equipment and idiosyncrasies.

 

William for president. Clarkson is a nonce!

[dschoem]

Hello there

Apologies for getting in on this question so late, despite Andrew's pleas ...

Reviewing is trickier than most people realise. Whether we like it or not, the reviewing process is largely subjective. And that would be the case even if we spent hours measuring components on a test bench. No matter how good or how bad a piece of kit measures, those figures will usually tell you very little about the actual sound.

Secondly, the reviewer has to contextualise the component, while considering it against some sort of reference. Placing an entry-level CD player in a high-end system may allow the shortcomings of the player to be  highlighted more effectively, but raises the question whether those shortcomings would ever become apparent in a more typical application.

I use a variety of systems for reviewing components submitted to AVSA - a so-called high-end stereo system, a smaller but still sonically critical system, and home theatre set-up. AVSA also has a properly prepared sound room in which some of the reviews are conducted.

The review kit we use is listed in the front of each issue of AVSA, but in my instance currently includes pre-amps and power amps by Viola Labs, Electrcocompaniet and Marantz, CD/SACD players by Esoteric and Rotel, speakers by Vivid Audio and Dynaudio, a Linn LP12/Ittok/Ortofon Kontrapunkt record deck, a Yamaha RX-V2700 AV amp, KEF AV speakers, a Hitachi plasma, Rotel DVD player ... and all sort of kit I've accumulated over the years. Contrary to popular belief, I own have paid for the majority of the stuff.

Part of the reasoning behind reference systems is that I know what the components in that system sound like, specifically in my room (which is a real-world room doubling as my study, by the way). By using separate 'high-end' and midrange systems, I do try to contextualise the application of a component, although I will often put a top-end unit in the smaller system, or vice versa, to cross-reference what I've heard, and to prevent a system anomaly to be ascribed to the component itself.

I agree that the only real benchmark in audio is the real thing: live music. Hi-fi is an attempt to reproduce the real thing, and the closer it gets to that objective, the better it is. However, what we actually hear and how we react to what we hear differs from individual to individual.

And so, the art of reviewing is keeping all these elements in mind, to never be prescriptive and to remind the reader that a review can only be a combination of information provision, advice and personal opinion.

William Kelly cuts straight to the chase on the latter point. I like what makes my feet tap, what makes me smile, what makes me spend hours and days rediscovering my music. Any component - even a cable - that can do that, is good in my books, regardless of price or technology. It's why I like vinyl and records, but not to the exclusion of other technologies ...

The reader gets to know the reviewer's styles, idiosyncracies and opinion, and decides on his own stance relative to the reviewer: at worst, what I like may be a flag to a reader to steer well clear!

Phew - a mouthful, I know, but there you have it. And thanks for inviting AVSA onto the forum - the success of any magazine relies on its contact with its existing and would-be readers. That communication is vital!

Deon Schoeman
AVSA

[Shonver]

Hi Deon

Welcome to the forum.

AVSA also has a properly prepared sound room in which some of the reviews are conducted.

and

Part of the reasoning behind reference systems is that I know what the components in that system sound like, specifically in my room (which is a real-world room doubling as my study, by the way).

My guess is that these rooms are quite well damped acoustically. But many middle-income homes have acoustically live rooms (tiled floors, bare walls & lounge suite). Would you consider testing (especially loudspeakers) in an environment more representative of the average home? I believe that this will show up many speakers (from well-reputed manufacturers) that sound good in a well-damped room.

I agree that the only real benchmark in audio is the real thing: live music. Hi-fi is an attempt to reproduce the real thing, and the closer it gets to that objective, the better it is. However, what we actually hear and how we react to what we hear differs from individual to individual.

I often wonder how realistic this benchmark is: most recordings are done in studios and then engineered to the nth degree. I agree that finally listening satisfaction is what counts.

[Gliding Dutchman]

I often wonder how realistic this benchmark is: most recordings are done in studios and then engineered to the nth degree. I agree that finally listening satisfaction is what counts.

Shaun - good point there...

One definite recording that is bare and best for review puposes is the NAIM True Stereo album (on Compact Disc). Got mine from Andries... and man(!!) this is as basic as recording can be but the detail and realism is something to behold. See: http://www.thenaimlabel.co.uk/true_stereo.htm

GD

[skinnyfat]

 And thanks for inviting AVSA onto the forum - the success of any magazine relies on its contact with its existing and would-be readers. That communication is vital!

And our thanks to you for accepting our invitation. It is of real value for our members and readers to have this access and contact with you guys at AVSA and we appreciate all the time you take to participate, I know how busy your days are.

Thanks Indeed
Greg
AVForums

[dschoem]

My guess is that these rooms are quite well damped acoustically. But many middle-income homes have acoustically live rooms (tiled floors, bare walls & lounge suite). Would you consider testing (especially loudspeakers) in an environment more representative of the average home? I believe that this will show up many speakers (from well-reputed manufacturers) that sound good in a well-damped room.

The AVSA room is quite well damped (it has to be in part to prevent from the other occupants in the building to tear out their hair!) but my home listening room is pretty much normal, and the AV system is in a very real-world lounge, with lots of glass, open spaces, tiles etc.

You're right that speakers will interact differently with different rooms - the point was made earlier that the listening room is as much a component of a system as any of the hardware. Of course, the way a system installed in such a room also plays a role - speaker positioning, in particular, is critical and can make the difference between great or indifferent performances, regardless of the quality of the electronics.

On the whole, however, a reviewer should be able to discern the performance trend of a speaker: an inherently bright speaker will sound bright, in comparative terms, even in a heavily damped room. The room will flatter the speakers, yes, but the reviewer should be factoring in what the potential impact the room will have - it's why it is important for the reviewer to limit system and room changes when evaluating an individual component.

Deon