Author Topic: Audio Science Review  (Read 3632 times)

Offline Michon

Re: Audio Science Review
« Reply #30 on: November 27, 2018, 08:31:15 AM »
Here's an informed opinion on ASR and the veracity of the measurements to be found there.

Take a moment and watch this Currawong (Amos Barnett) video from 34:55 to around 36:00

https://m.youtube.com/watch?v=zjF9HnCNJKU


 :popcorn:

I know next to nothing about about ASR and Amir.

Are there any more details about (or reference to) his appearance at an Audio Precision presentation requesting a unit that is easier to operate?

Offline shadow.clone

Re: Audio Science Review
« Reply #31 on: November 27, 2018, 11:37:04 AM »
Here's an informed opinion on ASR and the veracity of the measurements to be found there.

Take a moment and watch this Currawong (Amos Barnett) video from 34:55 to around 36:00

https://m.youtube.com/watch?v=zjF9HnCNJKU


 :popcorn:

https://en.wikipedia.org/wiki/Ad_hominem

If I was still involved with hardware development, I would also request easier to use test equipment. Pretty sure anyone would do so in any field with any equipment. The context of his question matters, otherwise ad hominem (i.e. the foundation of politics :tongue:).


Offline Michon

Re: Audio Science Review
« Reply #32 on: November 27, 2018, 12:07:07 PM »
https://en.wikipedia.org/wiki/Ad_hominem

If I was still involved with hardware development, I would also request easier to use test equipment. Pretty sure anyone would do so in any field with any equipment. The context of his question matters, otherwise ad hominem (i.e. the foundation of politics :tongue:).

This is partly why I ask. Amos does not go into detail except for saying that Amir is incompetent and mentions different measurement results from others, using a heat gun to heat components and an incident at an Audio Precision presentation as the reasons.

Is there a well written and evidence based critique of the measurements posted by ASR? If so, please link to it. I am not interested in a discussion about the correlation of measurements and perception, but rather the validity of his measurement methodology and the repeatability of its results.

I was made aware ASR earlier this year, but have yet to consume any of the content. If it can be shown that his measurements are fraught with error I will know not to waste my time.

Offline capetownwatches

Re: Audio Science Review
« Reply #33 on: November 27, 2018, 01:14:44 PM »
Is there a well written and evidence based critique of the measurements posted by ASR? If so, please link to it.

Personally I have no axe to grind with Amir at all, and in fact came to his defence when he appeared (briefly) on this very forum some time ago.

I started reading ASR about 2 years ago and liked (and still do, in theory) the science-based position Amir espouses.
That said, there are numerous instances where his measurements have contradicted some very experienced and respected posters' findings.

Here is just one: https://www.superbestaudiofriends.org/index.php?threads/measurements-are-hard.5159/

There are others, but I'm not going to do all the work for you lazy buggers!  :dop:

The problem I have with Amir's testing is that erroneous conclusions can lead to inexperienced folk making poor decisions.
Many don't have the time, inclination or technical knowledge to refute his findings, and might take them at face value and as fact.

How many potential buyers have given Audio-gd amps a wide berth WITHOUT LISTENING TO ONE following his test results, for example?
« Last Edit: November 27, 2018, 01:27:00 PM by capetownwatches »

Offline gLer

Re: Audio Science Review
« Reply #34 on: November 29, 2018, 04:26:58 PM »
I don't know Amir from a bar of soap, and still read the reviews on ASR from time to time (for the sake of completeness of course). But one line from Currawong's video sums up my feelings about ASR (and hardcore objectivists in general): Tone sweeps are NOT music.

Enough said. 
In Pursuit of Head-Fi Perfection
Views and reviews at www.theheadonist.com
Follow me on facebook.com/liketheheadonist

Offline naughty

Re: Audio Science Review
« Reply #35 on: November 29, 2018, 04:40:17 PM »
Tone sweeps are NOT music.

sure .... but music is a collection of mutiple tones and frequencies .... so tone/frequency sweeps and analyzing various individual tones/frequencies are a relevant way of testing equipment for the purpose of music playback .... the problem is that many people do not know how to interpret the results of this analysis properly  :nfi:

Offline amirm

Re: Audio Science Review
« Reply #36 on: December 17, 2018, 05:32:43 AM »
Right here:
https://www.audiosciencereview.com/forum/index.php?threads/budget-dac-review-schiit-modi-2-99.1649/

Amir unfortunately doesn't know his way around an analyzer, and those measurements have been shown to be erroneous.

Here are some contrasting measurements of the Modi 2:
https://www.superbestaudiofriends.org/index.php?threads/measurements-are-hard.5159/#post-165046

The above post gives some perspective. To quote Marv:

1. Measurements are a pain the in ass, and they are very easy to screw up.
2. Having expensive gear like an AP doesn't mean it's any easier getting them right.
3. Beware of rando measurements on the Internet.

Folks on SBAF post plenty of random measurements.  I have found issue after issue that is present in their measurements yet they finish it with, "oh, isn't this a great product?" 

I got so tired of addressing the nonsense they post that i wrote an article on it.  See: https://www.audiosciencereview.com/forum/index.php?threads/are-measurements-of-schiit-yggdrasil-dac-inconsistent.3812/

In case after case, I show that their measurements are actually consistent with mine.  And where they differ is because they were given special versions of the product under the table.

As to my credentials, here they are: https://www.audiosciencereview.com/forum/index.php?threads/a-bit-about-your-host.1906/

Now, post and compare their credentials to mine.  I bet you don't even know what their credentials are.

I know as non-technical people it may be hard for some of you to adjudicate these things.  But please don't put aside your sensibilities.  Look up the print ads from Schiit and see the references to folks you are choosing to believe.   Here is that bit from my thread above:



Does this read as someone who is impartial testing audio gear in dispute?

Let's remember that it is my work that eventually forced Schiit to go and buy a decent audio analyzer and produce good products like Schiit Modi 3. 


Offline amirm

Re: Audio Science Review
« Reply #37 on: December 17, 2018, 05:37:13 AM »
Haha, very possibly. Huge can of worms indeed! If you really think a $299 Topping DX7 is better than a $2399 Schiit Yggy because it 'measures better', good for you I say. You just saved upward of R20k on tech you don't need. Go take the family on holiday or something. Maybe that's the point anyway ;)
Good for him indeed.  Have you done a controlled blind test comparing the two?  I have.  If you can tell the difference between the two, run the test and if you can blindly tell them apart, I will send you USD $500 for your troubles.  Make a youtube video with someone making the AB switch, run a dozen times and we can talk.

Remember, Schiit has gone to the drawing board and fixed major issues like linearity in Yggdrasil as a result of my testing.  For the first time ever, they have started to post Audio Precision measurements for products such as Schiit Modi 3.  If measurements don't matter, why do they post these results now?  Think about that.  You are letting beliefs overcome data, facts and audio science.

A DAC has but one job: to faithfully convert digital samples to analog.  When it starts to do that incorrectly, then it is not a good DAC.  No way can you make a case that screwing up audio samples and generating distortion that is not in your music is "good for you."

Offline amirm

Re: Audio Science Review
« Reply #38 on: December 17, 2018, 05:44:45 AM »
What I do take issue with is how valid those measurements are in determining the overall sound quality/character of the dac/s in question, but more than that, the relative impact of that quality (or lack thereof) on the entire chain (which you described so well above).
That is not what I do.  I measure equipment to see if they are engineered well and do what the manufacturers claim.  Instrumentation together with interpretation of results quickly demonstrates the good from bad.  Bad designs are not there because someone thought those things were good ideas.  They are bad because the people who designed the gear either didn't know what they were doing, or lacked the equipment and knowledge to measure the gear to see if it is working right.

Most DACs today are using a DAC chip internally with superb specifications.  It is the implementation of that DAC chip that either preservers that performance, or screws it up.  Measurements are the absolute tool to show that.

Now, when problems are shown in measurements, I apply science of psychoacoustics as to whether they are audible problems or not.  So there are comments about audibility but based on sound science, not random folklore online.

And I have yet to hear a "character" in any DAC.  What you read about such things is all sourced in improper audio testing resulting in totally incorrect conclusions.  Do a level matched, controlled test of two DACs blind and then tell me one sounds warm and the other doesn't.  I have done this many, many times.  And I have done it sighted to know how wrong those conclusions are.

Ultimately what I bring to ASR Forum is the best of what audio science knows, and actual, concrete, repeatable data about performance of audio products.  This is causing a good change in the industry already, with companies modifying their products to fix things, designing new performant products, and in general taking proper design and verification seriously. 

Without it, we have wild west where any claims about audio fidelity works.  Guy says put a rock on your DAC and it sounds good and many people believe.  In what other area of your life are you this open to suggestions and dismissal of science?

Offline amirm

Re: Audio Science Review
« Reply #39 on: December 17, 2018, 05:49:59 AM »
It really depends on what you test. Normal test wave forms are usually quite simple, there is not too much you can read into them, but if it struggles to output simple wave forms you can bet it will struggle with more complex ones.
Sorry no.  If an audio product distorts one tone, then it will distort many tones worse.  I post 32-tone tests now in my reviews.  They show nothing new other than what was bad, is really bad.  :)  Here is an example from my review of Schiit Budget DACs: https://www.audiosciencereview.com/forum/index.php?threads/review-battle-of-schiit-audio-dacs.5487/

Schiit modi 3:


Schiit modi multibit:


See?  Other measurements of Schiit Multi-bit showed it to be much worse than Modi 3.  Multiple tones just multiplies the effect.

A DAC has no idea what it is being fed to it anyway.  It is all digital values to be converted to an analog voltage.  Spectrums like above are done with signal processing.  The DAC is dumb and does what it does to one tone that it does to many.

Offline amirm

Re: Audio Science Review
« Reply #40 on: December 17, 2018, 05:57:29 AM »
I don't know Amir from a bar of soap, and still read the reviews on ASR from time to time (for the sake of completeness of course). But one line from Currawong's video sums up my feelings about ASR (and hardcore objectivists in general): Tone sweeps are NOT music.

Enough said.
You don't know me from a bar of soap?  I need to get my name on some.  :D

Currawong needs to study audio science.  If he did, he would know that every lossy audio codec converts music to pure tones, applies compression there, and then converts it back when you play it.  If individual tones are not music, how come your MP3 sounds like music?  Of course Currawong and countless others who say these things are not technical and have no experience in signal processing which begs the question: why do you guys believe them?  Why not ask him for his credentials and ask him if he knows of any paper at Audio Engineering Society, ASA, IEEE, etc. backs what he says?

Folks, it is time that we all graduate from this folklore.  We can't keep believing lay people just because they do reviews.  There are decades of audio science that have answered all of this.  We don't get to invent new audio science on our own, and spend our money on them as if they are correct.  They are not. 

Let's not keep creating FUD and doubt around good data.  Let's encourage great engineering and design.  Why do we want audio to be excluded this way?

Offline stereosane

Re: Audio Science Review
« Reply #41 on: December 18, 2018, 07:07:37 AM »
But in the end measurements mean nothing if the sound isnít good, and Iíve got an item that measures very bad yet is the best sounding component Iíve owned. Fair enough good electrical work should measure within specs usually but what if it sounds better by changing some of those parameters, is it still wrong or a bad design I donít think so.. as long as it sounds good to me, there no one right sound, everyone hears things slightly different so perfect measurements and bad measurements will all sound different to different people..

I truly think thatís why so many people upgrade, bought an item because it reviewed well and specs are the best yet somehow when listening to it you just donít feel the magic, like my Nad M51 Dac.. very very good test results yet I found it a bit sterile and cold, my Audio-GD Dac sounds like heaven to my ears compared to that Nad yet the Audio-GD measures poorly against it..

Its not about trusting your ears itís about letting your ears hear how they hear which is very unique to each person :)


Offline gLer

Re: Audio Science Review
« Reply #42 on: December 18, 2018, 07:44:49 AM »
But in the end measurements mean nothing if the sound isnít good, and Iíve got an item that measures very bad yet is the best sounding component Iíve owned. Fair enough good electrical work should measure within specs usually but what if it sounds better by changing some of those parameters, is it still wrong or a bad design I donít think so.. as long as it sounds good to me, there no one right sound, everyone hears things slightly different so perfect measurements and bad measurements will all sound different to different people..

I truly think thatís why so many people upgrade, bought an item because it reviewed well and specs are the best yet somehow when listening to it you just donít feel the magic, like my Nad M51 Dac.. very very good test results yet I found it a bit sterile and cold, my Audio-GD Dac sounds like heaven to my ears compared to that Nad yet the Audio-GD measures poorly against it..

Its not about trusting your ears itís about letting your ears hear how they hear which is very unique to each person :)
Youíve just hit the nail on the proverbial head. The idea that every amp and dac should just do its job and get out the way of the music is all good and well, but doesnít even start to explain why two components that supposedly do that sound completely different to each other. And we also tend to forget that the biggest variable of all is the transducer, but a very large margin too. You could have the worldís most Ďperfectí, best measuring amp and it will still sound like cr** with a rubbish transducer.

Bottom line: be guided by others with experience (and by that I mean listening experience, not qualifications - there are no Ďqualificationsí for knowing what to listen for) but only ever trust your own ears.
In Pursuit of Head-Fi Perfection
Views and reviews at www.theheadonist.com
Follow me on facebook.com/liketheheadonist

Offline amirm

Re: Audio Science Review
« Reply #43 on: December 18, 2018, 09:15:43 PM »
But in the end measurements mean nothing if the sound isnít good, and Iíve got an item that measures very bad yet is the best sounding component Iíve owned.

[...]

Its not about trusting your ears itís about letting your ears hear how they hear which is very unique to each person :)
You have no way of assessing what your ears received.  What you perceived is the combination of what the ear heard, and your brain decided.  The latter is highly adaptive. When listening to music for example, it is discarding 99% of what the ear is hearing.  Can you imagine it capturing every note, every nuance, every bit of sonic signal?  You would need a brain the size of everyone's combined.  :)

So what happens is that a highly lossy process is applied to your hearing when you enjoy music.  This is called translation from short-term/echoic to long-term memory.

When people perform tests of new product, they act different.  They use their brain to analyze fidelity.  Then they will hear detail, air, etc. that was always there but was missed before.  Quickly then they form an opinion that this new gear must be sounding better.  This is the case whether the sound waves changed one bit, or whether the brain received anything different!

This is why we don't test audio gear in listening tests like you all do.  We perform the test blind.  That way, the above analysis cannot benefit from knowing what is new gear and what is not.

When we perform the correct test as above, then the differences you perceive go out the window.  Folks can't hear the clear, night-and-day differences they thought existed before.

This is an easy test to do: play the same digital file multiple times in a row.  Then focus on differences, in detail, etc.  I guarantee you that you will hear different fidelity in every play even though you know for a fact there is no difference!  Your brain is that good at confusing you.

In sharp contrast, measurements are reliable.  When something distorts, it is guaranteed to screw up the fidelity of music you bought.  I have yet to find a case of a unit distorting yet sounding better in controlled testing. 

Controlled testing by the way also includes matching levels and making sure channels are not swapped.  Time after time, I have heard clear fidelity differences that simply vanish like a fart in the wind when I match levels.  Even when testing sighted these controls go a long way toward producing more reliable results.

Note that what I am writing here is the opinion of professional research community in audio.  It has been developed over decades of research, experimentation, knowledge of our hearing and brain, etc.

As I said, you are welcome to invent your own audio science but please don't put it forward as having any validity.  We (audio scientist and engineers) are not stupid to spend so much extra work to run controlled tests if all we had to do was test things as you do.  We do it because it is *essential* for generating correct results.

As the movie matrix, ultimately you can continue to live in the illusion of matrix or adopt what the real world/science is.  I am here to pursue and advance the latter.

Offline amirm

Re: Audio Science Review
« Reply #44 on: December 18, 2018, 09:24:43 PM »
Youíve just hit the nail on the proverbial head. The idea that every amp and dac should just do its job and get out the way of the music is all good and well, but doesnít even start to explain why two components that supposedly do that sound completely different to each other.
Once again, to say two things sound the same requires that you only judge the sound.  You have no ability to do that in typical audiophile tests.  Your brain is a key part there and it manufactures conclusions that are hugely wrong and have been shown to be the case numerous times.  I have done this work professionally and I can tell you your family's livelihood depends on knowing the correct facts, you never perform audio tests the way audiophiles do.

There is a reason not one DAC manufacturer has ever posted a blind/controlled test showing their stuff sounds better.  I on the other hand, have done this so many times.  I guarantee you that what you think sounds different, will not at all hold up to scrutiny in a controlled test.  Your conclusions are illusions. 

And the problem with illusions is that they won't last.  This is why audiophiles keep upgrading things.  Every time they evaluate another audio gear, they arrive at a random conclusion not backed by what their hearing is capable of.  So they routinely decided the new gear is better so buy that product.  And advocate it until they see the next thing they think is better.

How many times do we hear they have bought a new gear and it removed a "veil?"  Seemingly infinite amount of veils can be removed.  Surely no logical person can believe that.

Quote
And we also tend to forget that the biggest variable of all is the transducer, but a very large margin too. You could have the worldís most Ďperfectí, best measuring amp and it will still sound like cr** with a rubbish transducer.

Bottom line: be guided by others with experience (and by that I mean listening experience, not qualifications - there are no Ďqualificationsí for knowing what to listen for) but only ever trust your own ears.
Absolutely trust your ears: just figure out how to get the brain and all the other factors it includes out of the way. This is what controlled tests do.  Otherwise you are creating a free-form illusion that is just not grounded in anything real.