3DMark cheats ATi ed Nvidia siamo al ridicolo...

[rubex]

What Is the Performance Difference Due to These Cheats?

"A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320.

The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks – a 24.1% drop.

Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further. "

La notizia completa:
http://www.warp2search.net/article.php?sid=12390&mode=thread&order=0

Quanto fumo negli occhi ci buttano,con i vari bench....e giu' a scannarsi con ati vs nvidia,c'e' da riflettere IMHO.

BYe

[checo]

questo conferma che il 3dmark non vale una cippa

[JCD]

conferma anche che c'è una bella differenza tra ati e nvidia..... il 24% è un po' diverso dal 2% scarso che potrebbe essere imputabile anche ad un'ottimizzazione dei driver verso una maggiore stabilità, imho..... che poi il 3dmark non serva a nulla, siamo (quasi) tutti d'accordo

[WhiteBase]

Originally posted by "rubex"

What Is the Performance Difference Due to These Cheats?

"A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320.

The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks – a 24.1% drop.

Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further. "

La notizia completa:
http://www.warp2search.net/article.php?sid=12390&mode=thread&order=0

Quanto fumo negli occhi ci buttano,con i vari bench....e giu' a scannarsi con ati vs nvidia,c'e' da riflettere IMHO.

BYe

L'8% di ati è concentrato in un singolo test... che influisce complessivamente per l'1,9% sul punteggio finale (margine irrisorio)... quello che si evince dal pdf di 7 pagine della futuremark (dove vengono spiegati nel dettaglio i cheat scandalosi di nvidia) è che comunque il rendering delle ati, anche nel test in questione (nelle fx passa da 37fps a 19, nelle 9800pro da 35 a 33), è corretto, a differenza di quello di nvidia che addirittura arriva a renderizzare solo le parti di scena mostrate dalla telecamera fissa e a sostituire completamente gli shader dell'acqua con altri che da quell'angolazione prefissata risultano molto simili agli originali, ma che con il 3dmark che hanno i tester saltano subito all'occhio, dato che usa una telecamera mobile comandabile dall'utente: con la fx scompaiono, perchè non renderizzate, intere sezioni di scena, con ati nulla di tutto questo, inotre non sono stati riscontrati cali di qualità.
Quello che fa pensare l'1.9% e su cui si sta investigando è la possibilità di una specifica ottimizzazione dei drivers (che la stessa Futuremark ritiene legittima e accettabile, completamente diversa dal cheating) o di un comportamento anomalo dei drivers.
Tra l'altro a quanto riporta Futuremark un cheating della funzione benck dei giochi più diffusi è persino più facile da realizzare e più difficile da scoprire rispetto a quella dei test sintetici

[dieg]

[Legolas84]

Scandaloso!

[andreamarra]

La dura legge del marketing, ovvero: scandaloso.

[SPhinX]

Non sapevo si potesse cheattare anche su queste cose, Ridicoli!

[prova]

CHEATER!

[Raven]

... sul forum nvitalia c'è un bel thread che ne parla approfonditamente (la 5800 ultra è scesa da 5300/5400 a 4900, dopo la patch...)... vi consiglio di leggerlo!

[pandyno]

Dai primi test le 9700 perdono dai 50 ai 70punti...

[Dragon2002]

Quindi il vantaggio di Nvidia nei confronti di Ati era tutto fasullo,la superiorita che ha avuto con gli ultimi driver anche per quanto riguarda la fx5800 e nn solo per la fx5900 era tutta una truffa,veramente incredibile,qui nn si tratta di provare a barare al 3dmark facendolo andare a 16bit o abbassando leggermente il dettaglio,ma adirittura scelgono cosa renderizzare e cosa nn renderizzare cosi facendo e come se la fx5800 e la fx5900 visualizessero un video,più che una bench in 3d in tempo reale,veramente assurdo cio che si fa per avere punteggi più alti nei bench e veramente assurdo che ci provino con madonion visto che è gia la seconda volta che li becca,nella prima i driver Nvidia tagliavano le espolosioni dal primo test degli aerei. Quindi ora che si fa,MadOnion dice sospetta che Nvidia ha barato anche sui giochi possibile e allora bisogna aspettare una release di driver senza trucchi e centinaia di siti e riviste di tutto il mondo devono rimangiarsi quello che di buono hanno detto sulla fx5900 e rifare la recensione da zero.
Assurdooooooooooooo

[pandyno]

http://www.hothardware.com/hh_files/S&V/r9800256mb_gffx5900upd(2).shtml

Copiate e incollate il link

[Dragon2002]

Ma è assurdo ora la fx5900ultra le prende al 3dmark è alla pari con tutti gli altri test alla 9800pro e vince solo in un test di ut2003,poi c'è da dire che primeggia quasi sempre anche con l'AA attivato anche se tutti sappiamo che l'AA4x della fx nn vale in qualita quello della 9800pro,in conclusione la 9800pro è tornata leader,Nvidia dopo l'fxflow ne ha fatto una ancora più grossa inventandosi i driver truccati.
Bisogna modificare migliaia di recensioni in tutto il mondo,perchè Nvidia ha truccato tutti i bench e ora chi se la compra sta scheda

[Darkman]

Avevo aperto un topic anche io. Lo faccio chiudere e spostiamo qui!

Dopo l'esclusione dei detonator beta modificati che non vengono più riconosciuti dalla Futurmark nel suo database, adesso siamo arrivati anche alla prima patch per il 3D Mark 2003.

La potete scaricare qui.

I nuovi detonator Fx sono stati creati apposta per "barare" al 3D Mark basandosi su un o stratagemma alquanto stupido quanto geniale.
Praticamente, essendo il 3dMark un rendering 3d a visuale fissa, tali detonator non disegnavano affatto tutti i poligoni non visibili dall'utente.
Tale applicazione è soltanto in parte applicabile anche ai giochi (ricordo l'HyperZ e l'HSR 3dfx oppure anche tutte le feature introdotte con il KyroII e le recenti GF). Una aplicazione del simile a quella nVidia con gli ultimi detonator avrebbe portato a artefatti mostruosi nei giochi 3d. Qualcuno forse avrà presente cosa succedeva mettendo i valori di HSR nella Voodoo5 al massimo: sparizione casuale di poligoni, errori nel loro posizionamento ecc...
Praticamente nVidia otteneva un valore così alto non grazie allo sforzo di ottimizzazione oppure per la potenza reale delle sue schede, ma mediante uno stratagemma.
Il bluff è stato scoperto usando una versione di sviluppo del 3dMark 2003 che permetteva la navigazione nel rendering della scena spostando a piacimento la visuale.

Certe una cosa mi viene in mente: ma nVidia non ha altro da fare??? Non mi sembra questo il metodo migliore per guadagnare acquirenti.

In our testing, all identified detection mechanisms stopped working when we altered the benchmark code just trivially and without changing any of the actual benchmark workload. With this altered benchmark, NVIDIA’s certain products had a performance drop of as much as 24.1% while competition’s products performance drop stayed within the margin of error of 3%. To our knowledge, all drivers with these detection mechanisms were published only after the launch of 3DMark03. According to industry’s terminology, this type of driver design is defined as ‘driver cheats’.
We are publishing this document to report to our customers in detail about our findings. Main reason behind publishing this document is to answer the criticism presented against synthetic benchmarks and their reliability when testing hardware performance. The document follows a question/answer format.

Naturalmente anche ATi è sotto investigazione:

A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with
3DMark03 build 320.
The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in
that build gets 4679 3DMarks – a 24.1% drop.
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on
this new build of 3DMark03. The drop in performance on the same test system with a Radeon
9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to
8.2% difference in the game test 4 result, which means that the test was also detected and
somehow altered by the ATI drivers. We are currently investigating this further.

[Darkman]

Beccatevi il pdf Futurmark!

Page 1 of 7
Futuremark Corporation
May 23rd, 2003
To the Media, Futuremark’s customers and business partners:
Audit Report: Alleged NVIDIA Driver Cheating on 3DMark03
After the launch of 3DMark03 Build 320 Futuremark has received reports from the members of its
BETA Program concerning certain anomalies with 3DMark03 and Nvidia drivers. ExtremeTech
(www.extremetech.com) has published an article1 on suspecting NVIDIA drivers to improperly
boost scores on Futuremark’s 3DMark®03. Some of these anomalies have also been reported by
Beyond3D2. Alarmed by all these reports Futuremark has conducted a thorough internal audit
regarding this matter and has verified that certain NVIDIA drivers indeed seem to have detection
mechanisms, which are triggered by components of the 3DMark03 program. We have identified
eight such mechanisms.
In our testing, all identified detection mechanisms stopped working when we altered the
benchmark code just trivially and without changing any of the actual benchmark workload. With
this altered benchmark, NVIDIA’s certain products had a performance drop of as much as
24.1% while competition’s products performance drop stayed within the margin of error of 3%. To
our knowledge, all drivers with these detection mechanisms were published only after the launch
of 3DMark03. According to industry’s terminology, this type of driver design is defined as ‘driver
cheats’.
We are publishing this document to report to our customers in detail about our findings. Main
reason behind publishing this document is to answer the criticism presented against synthetic
benchmarks and their reliability when testing hardware performance. The document follows a
question/answer format.
How Were These Driver Cheats Found?
Members of Futuremark’s BETA program3 first noticed how parts of the tests in 3DMark03 were
rendered differently on different hardware. When testing NVIDIA hardware on 3DMark03 with socalled
developer’s version’s free camera enabled, they noticed how some parts of tests were
rendered strangely, and informed Futuremark of their findings. Futuremark investigated further
and our findings show that certain NVIDIA drivers seem to detect when 3DMark03 is running and
then replace the 3DMark03’s rendering requests with manually implemented alternative rendering
operations. These alternative rendering operations reduce the amount of rendering work and
thereby increase the obtained benchmark result.
1 Extremetech: http://www.extremetech.com/article2/0,3973,1086025,00.asp
2 Beyond3D: http://www.beyond3d.com/#news5856
3 Futuremark’s BETA program is an open, fee based cooperation program between Futuremark
and the PC industry at large. BETA program members have access to pre-release builds of
upcoming benchmarks and to a so-called developer build. The developer build is exactly the
same as the public version of the benchmark, but with additional functionality. Amongst other
things, the developer build has a ‘free camera’ mode, where the user can manually move the
camera around while the test is running.
Page 2 of 7
Why Does This Matter – It Is Just a Synthetic Benchmark?
We acknowledge with great pride how big a role 3DMark has in the PC industry. 3DMark score
has become perhaps the most influential metric of PC performance. Enthusiasts, professional
hardware reviewers and OEMs all depend on 3DMark results to a great extent.
We have a tremendous responsibility towards our users, who count on us and on our products
when making important decisions. Thus, it matters a great deal that no one is able to take
advantage of 3DMark – or any other significant benchmark - with unfair means.
Well designed synthetic benchmarks are excellent tools to objectively compare performance and
to reveal different architectures’ strengths and weaknesses. Some commentators have argued for
only using benchmarks based exclusively on games; however there are severe problems with
relying only on this approach:
· Game benchmarks only demonstrate how the hardware performs for that particular game and
do not indicate the overall performance of the hardware.
· Any cheats potentially included in drivers are much easier to hide in game benchmarks.
· Finally, synthetic benchmarks can stress the hardware in a variety of ways, allowing
reviewers to explore performance in particular areas and extrapolate performance of a
hardware for upcoming titles.
Why Is It More Difficult to Cheat In a Synthetic Benchmark?
Actually, it may not be more difficult to cheat in synthetic benchmarks than it is in benchmarks
embedded in games. But one thing is for sure, it is much more difficult to cheat in synthetic
benchmarks without being caught.
The reason is that a synthetic benchmark does not exist in isolation, but as part of a set of tools.
These tools can be used to analyze the benchmark run for irregularities. Further, the tools can be
used to expose the existence of the cheats – both ExtremeTech and Beyond3D have proven this
point very effectively. Game benchmarks are not immune to cheating either. Just like drivers can
identify when 3DMark03 is running, same drivers also can identify when a popular game’s
benchmark is running. The driver can even detect when the benchmark tool of the game is
running to enable the cheats; the cheats are then disabled when the actual game is played. Thus,
the benchmark portion can be made to run faster with cheats while the actual in-game play
performance is not enhanced at all.
Game benchmark cheats are much more difficult to expose. Even with the presence of clues,
such as reduced image quality or missing features, it is next to impossible to prove what is going
on without the source code and game developers’ active cooperation. The major problem with this
is following: There are few if any game developers that want to reveal their intellectual property to
third parties so that independent groups would be able to prove the cheats.
Furthermore, game developers are in the business of making games and they want to attract as
broad an audience as possible for their game. Their success does not depend on making sure
that potential cheats on their benchmarks are exposed, whereas it is a benchmark house’s
business actively audit drivers and expose any cheating.
Page 3 of 7
Aren’t These Cheats Just Optimizations That Also Benefit General Game Play
Performance?
No. There are two reasons.
Firstly, these driver cheats increase benchmark performance at the expense of image quality.
Only the user and the game developer should decide how a game is meant to be experienced,
and not the hardware developer. An act by hardware developer to force a different experience
than the developer or the user intended, is an act that may mislead consumers, the OEMs and the
media who look to our benchmark to help them make purchase decisions.
Secondly, in well-designed benchmarks like 3DMark03, all cards are instructed to do the same
amount of work. Artificially reducing one card’s workload, for example, by using pre-set clip planes
or using a lower precision shader against the program’s instructions, is only aimed to artificially
manipulate the benchmark test result. Please note, that the cheating described here is totally
different from optimization. Optimizing the driver code to increase efficiency is a technique often
used to enhance game performance and carries greater legitimacy, since the rendered image is
exactly what the developer intended.
What Are The Identified Cheats?
Futuremark’s audit revealed cheats in NVIDIA Detonator FX 44.03 and 43.51 WHQL drivers.
Earlier GeForceFX drivers include only some of the cheats listed below.
1. The loading screen of the 3DMark03 test is detected by the driver. This is used by the driver
to disregard the back buffer clear command that 3DMark03 gives. This incorrectly reduces the
workload. However, if the loading screen is rendered in a different manner, the driver seems
to fail to detect 3DMark03, and performs the back buffer clear command as instructed.
2. A vertex shader used in game test 2 (P_Pointsprite.vsh) is detected by the driver. In this case
the driver uses instructions contained in the driver to determine when to obey the back buffer
clear command and when not to. If the back buffer would not be cleared at all in game test 2,
the stars in the view of outer space in some cameras would appear smeared as have been
reported in the articles mentioned earlier. Back buffer clearing is turned off and on again so
that the back buffer is cleared only when the default benchmark cameras show outer space.
In free camera mode one can keep the camera outside the spaceship through the entire test,
and see how the sky smearing is turned on and off.
3. A vertex shader used in game test 4 (M_HDRsky.vsh) is detected. In this case the driver adds
two static clipping planes to reduce the workload. The clipping planes are placed so that the
sky is cut out just beyond what is visible in the default camera angles. Again, using the free
camera one can look at the sky to see it abruptly cut off. Screenshot of this view was also
reported in the ExtremeTech and Beyond3D articles. This cheat was introduced in the 43.51
drivers as far as we know.
4. In game test 4, the water pixel shader (M_Water.psh) is detected. The driver uses this
detection to artificially achieve a large performance boost - more than doubling the early
frame rate on some systems. In our inspection we noticed a difference in the rendering when
compared either to the DirectX reference rasterizer or to those of other hardware. It appears
the water shader is being totally discarded and replaced with an alternative more efficient
shader implemented in the drivers themselves. The drivers produce a similar looking
rendering, but not an identical one.
5. In game test 4 there is detection of a pixel shader (m_HDRSky.psh). Again it appears the
shader is being totally discarded and replaced with an alternative more efficient shader in a
similar fashion to the water pixel shader above. The rendering looks similar, but it is not
identical.
6. A vertex shader (G_MetalCubeLit.vsh) is detected in game test 1. Preventing this detection
proved to reduce the frame rate with these drivers, but we have not yet determined the cause.
Page 4 of 7
7. A vertex shader in game test 3 (G_PaintBaked.vsh) is detected, and preventing this detection
drops the scores with these drivers. This cheat causes the back buffer clearing to be
disregarded; we are not yet aware of any other cheats.
8. The vertex and pixel shaders used in the 3DMark03 feature tests are also detected by the
driver. When we prevented this detection, the performance dropped by more than a factor of
two in the 2.0 pixel shader test.
We have used various techniques to prevent NVIDIA drivers from performing the above
detections. We have been extremely careful to ensure that none of the changes we have
introduced causes differences in either rendering output or performance. In most case, simple
alterations in the shader code – such as swapping two registers – has been sufficient to prevent
the detection.
What Is the Performance Difference Due to These Cheats?
A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with
3DMark03 build 320.
The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in
that build gets 4679 3DMarks – a 24.1% drop.
Our investigations reveal that some drivers from ATI also produce a slightly lower total score on
this new build of 3DMark03. The drop in performance on the same test system with a Radeon
9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to
8.2% difference in the game test 4 result, which means that the test was also detected and
somehow altered by the ATI drivers. We are currently investigating this further.
Why does this matter?
The latest version in the 3DMark series, 3DMark03, was developed for performance
measurement and comparison for the DirectX 9 generation of hardware. All major graphics
hardware and CPU manufacturers participated in specifying and testing 3DMark03, in order to
ensure that it is an impartial tool that correctly measures the PCs performance in real-time 3D
applications, like 3D games. The IHVs see the existence of such an impartial tool so important
that they participate in the funding of 3DMark development, by joining the Futuremark beta
program.
Games strive to above all offer the best gaming experience, which many times means, that the
game needs to be run differently on different hardware. This in turn means that many game
benchmarks run differently on different hardware, or in other words submit a differing workload on
different hardware. This kind of game benchmark does measure how fast that game runs, but it
does not offer a general hardware performance comparison.
3DMark on the other hand is designed for this very purpose. If the drivers do their job correctly,
3DMark produces the precisely same rendering on all hardware. Some hardware with a more
limited feature set actually do more work in order to achieve the same rendering, but the resulting
rendering is the same, the workload is comparable, and the performance result is justified as a
world wide standard.
Is This Just Futuremark Taking Revenge on Nvidia for Leaving the BETA
Program?
No. Our only purpose is to maintain the integrity of our products, and so ensure continuity of our
business. We treat the BETA program and the whole industry very seriously; Beta Program
members include leading companies of the industry such as Microsoft, Dell, AMD and Intel.
Our business relies on our ability to produce unbiased benchmarks that enable apples-to-apples
comparisons. Any company that deliberately tries to fool our products is actually trying to take an
advantage of our credibility. We will not tolerate such actions from anyone.
Page 5 of 7
What Happens Now?
When 3DMark03 is altered slightly, NVIDIA drivers do not recognize 3DMark03 anymore, and the
performance drops. The same slightly altered 3DMark03 version can be run on other hardware
and the results remains the same.
Futuremark has prepared a second patch for 3DMark03, Build 330, where these preventative
changes are implemented. We hope this will encourage the whole industry to release drivers that
render the tests as 3DMark03 intends them to be rendered, and which produce results that can be
fairly compared to those obtained with other hardware. It is possible that new drivers that cheat
also in the new 3DMark03 build may be released in the future. We sincerely hope this will not
happen. We strive to provide all BETA program members with robust tools to detect such
situations.
Can 3DMark03 be used as a reliable benchmark for DirectX 9 generation
graphics cards?
Yes, with the new 3DMark03 build 330, it can. We recommend benchmarking GeForceFX cards
using the NVIDIA drivers 44.03 or 43.51, and the new 3DMark03 build 330. We also recommend
benchmarking Radeon 9000 series cards using the ATI Catalyst 3.4 drivers, and the new
3DMark03 build 330. These combinations run the 3DMark03 tests correctly, according to how
3DMark03 instructs the hardware to render the 3D scenes.

[SPhinX]

Originally posted by "pandyno"

http://www.hothardware.com/hh_files/S&V/r9800256mb_gffx5900upd(2).shtml

Copiate e incollate il link

E' sempre +una buffonata, si stanno arrampicando sugli specchi e fare una cosa del genere è molto scorretto!!! Ma con quale faccia possono fare una cosa del genere??? Proprio non lo capisco!!

[rubex]

Originally posted by "pandyno"

http://www.hothardware.com/hh_files/S&V/r9800256mb_gffx5900upd(2).shtml

Copiate e incollate il link

Incredibile!!!
Quasi 1200 punti in meno al 3D03 la 5900 ultra

VERGOGNA NVIDIA!!

[m4gnum]

Originally posted by "Darkman"

Avevo aperto un topic anche io. Lo faccio chiudere e spostiamo qui!

Dopo l'esclusione dei detonator beta modificati che non vengono più riconosciuti dalla Futurmark nel suo database, adesso siamo arrivati anche alla prima patch per il 3D Mark 2003.

La potete scaricare qui.

è poi la secondo patch

[prova]

Leggendo tutto il materiale che avete postato [c'è da perdere il sabato mattina] la prima reazione che si ha è di sconcerto, seguito poi da incredulità e infine da una sana dose di disappunto condito di rabbia.

Ma PORCA DI QUELLA PUTTANA! Io spendo 600 [SEICENTO!!!] EURO, che è probabilmente più di quello che una famiglia media riesce a mettere da parte alla fine del mese, per un prodotto! Le spendo perchè attirato dalle promesse di elevate prestazioni fatte da un marketing sempre più aggressivo, e poi vengo a sapere che OLTRE a farmi pagare 600euro tentano ANCHE di PRENDERMI PER IL CULO?!!!!

Sono indignato.
E mi piacerebbe che queste parole le leggesse qualcuno responsabile di questa vera e propria TRUFFA!

E se fossimo in una società con le palle, qui ci starebbe TUTTA una denuncia per pubblicità ingannevole o l'equivalente, dal momento che noi consumatori per valutare un prodotto del genere non abbiamo altra possibilità che affidarci ai test che vengono compiuti.
E se questi test vengono alterati noi siamo TRUFFATI, perchè acquistiamo un prodotto sulla base di promesse e prestazioni che NON CI SONO!

Mi piacerebbe davvero che si montasse un vero casino su questa roba.
Ma so già benissimo che passerà in sordina, dal momento che non esiste nessuna norma che condanni un simile comportamento sleale.

Incazzatissimo,
*Eroes*