"Knowing the identity behind a crawler tells us relatively little about whether its behavior is harmful. Under the New York bill, a company that discloses its identity and then accesses the website 10,000 times a minute complies with the law even though it is still imposing enormous and likely unjustified costs on the site. But a journalist anonymously using a crawler to investigate potentially anti-competitive practices by tech companies does not."
It does seem like this bill mostly affects normal users and is suspect with more surveillance being pushed. However the biggest problem is that the internet is global and the fees only apply to the US. If bots from Russia, China, India, or even France are scrapping then how would this even reduce the bots? Seems more like a new 'contempt of business model' is the purpose of this bill.
> In 2022, Tech Policy Press received a grant from the Omidyar Network for a research project evaluating vulnerabilities in the design, engineering, and use of encrypted messaging applications.
https://omidyar.com/ is geofenced for me, presumably because they identified me as being anti open web.
The article is ai lobby intended to repel a bill that would
> allow the FTC to pursue penalties of up to $53,000 if a person uses a stealth bot in a way that is “reasonably likely to damage, impair, or burden” a website’s “technical or commercial operation.”
The definition of "stealth bot" is a bot that does not identify itself as such and their main grief to the proposal is
> If researchers cannot easily understand what it means to “deploy” a crawler in a prohibited way, or what activity is “reasonably likely to burden or impair a website,” they may refrain from scraping rather than risk violating the law.
Which is the biggest load of shit I've heard given that all you need to do is identify yourself as a bot and respect robots.txt or be banned, the status quo before a bunch of assholes started systematically destroying the internet.
You can still ban everyone who doesn’t respect robots.txt. My experience with a crawler is that people don’t actually provide a robots.txt far more often. I can back off on 429 but I’d rather use Crawl-Delay etc.
Bots have limited ability to announce themselves. I just use fixed IP range and UA. But the Chinese trawlerbots use smart TV residential proxies and will copy someone else’s UA.
Any bot is operating on behalf of one or more humans somewhere. Bots are good, for the most part.
The conventional way of doing web isn't efficient. If it were, no one would be complaining about bots. The way we do web is the thing that needs to be updated.
What gets me is how much the people who advocate for the "open web" have completely incorporated the values of Facebook and Google.
Like there's that anime girl who harasses you when you visit so many open source web sites that just continues the pattern of being harassed by cookie dialogs and consent dialogs and pop-up ads and being asked for your email address and of course there is Wikipedia which is doing more and more of the same.
Another web is possible but web populists don't really believe it.
Just to get started, bots have been better at filling out CAPCHTAs than humans for a long time. I knew someone who got his site hacked by Russians circa 2008 because he published a neural network in freakin PHP that could read them. Anubis causes humans stress, slows down task completion, probably raises peoples blood pressure, and cumulatively causes a great deal of harm to users the same way Cloudflare's system does. Bots can use async or virtual threads and not even have to hold on to a stack while they are waiting for this bullshit, clicking on buses, etc.
CAPTCHAs, cookie banners, and of the that crap should be an automatic F grade on WCAG and should make web sites immediately liable for ADA lawsuits with triple damages.
>"In 2022, a Pulitzer-winning Wall Street Journal investigation analyzed over 31,000 financial disclosures from 12,000 senior government employees, revealing that nearly a quarter of federal officials traded stock in industries their agencies regulated."
Ya know, on the one hand, I wonder how many federal officials own nVidia stock and stock of other AI companies...
But then again, on the other, I wonder how many nVidia employees (and those of other AI companies!) own government T-bills (and other government backed securities!) in their 401K or other investment accounts!
You know, everybody in large corporations and governments owning shares in other large corporations and governments -- adds a whole new level of meaning to "stakeholder capitalism"... :-)
(In the far future, there'll probably be small foreign governments which become their own ticker symbol -- and get publicly traded on various stock exchanges! :-) You know, like: "Would you like to buy a share in Denmark (DNMK) or perhaps Norway (NRWY)? I hear they're very competitive with Apple (AAPL) right now! Great price toearnings (PE) ratio!" :-) )
You can already buy shares in a nations economy - it is called owning currency. The big difference from stocks is that most countries issue new shares (dollars, etc) to keep the price stable (indirectly via careful control of interest rates).
I am simplifying terribly but just as shares represent a fraction of a company, cash represents a share of a nation’s economy.
It does seem like this bill mostly affects normal users and is suspect with more surveillance being pushed. However the biggest problem is that the internet is global and the fees only apply to the US. If bots from Russia, China, India, or even France are scrapping then how would this even reduce the bots? Seems more like a new 'contempt of business model' is the purpose of this bill.
https://omidyar.com/ is geofenced for me, presumably because they identified me as being anti open web.
The article is ai lobby intended to repel a bill that would
> allow the FTC to pursue penalties of up to $53,000 if a person uses a stealth bot in a way that is “reasonably likely to damage, impair, or burden” a website’s “technical or commercial operation.”
The definition of "stealth bot" is a bot that does not identify itself as such and their main grief to the proposal is
> If researchers cannot easily understand what it means to “deploy” a crawler in a prohibited way, or what activity is “reasonably likely to burden or impair a website,” they may refrain from scraping rather than risk violating the law.
Which is the biggest load of shit I've heard given that all you need to do is identify yourself as a bot and respect robots.txt or be banned, the status quo before a bunch of assholes started systematically destroying the internet.
Bots have limited ability to announce themselves. I just use fixed IP range and UA. But the Chinese trawlerbots use smart TV residential proxies and will copy someone else’s UA.
The conventional way of doing web isn't efficient. If it were, no one would be complaining about bots. The way we do web is the thing that needs to be updated.
Like there's that anime girl who harasses you when you visit so many open source web sites that just continues the pattern of being harassed by cookie dialogs and consent dialogs and pop-up ads and being asked for your email address and of course there is Wikipedia which is doing more and more of the same.
Another web is possible but web populists don't really believe it.
Just to get started, bots have been better at filling out CAPCHTAs than humans for a long time. I knew someone who got his site hacked by Russians circa 2008 because he published a neural network in freakin PHP that could read them. Anubis causes humans stress, slows down task completion, probably raises peoples blood pressure, and cumulatively causes a great deal of harm to users the same way Cloudflare's system does. Bots can use async or virtual threads and not even have to hold on to a stack while they are waiting for this bullshit, clicking on buses, etc.
CAPTCHAs, cookie banners, and of the that crap should be an automatic F grade on WCAG and should make web sites immediately liable for ADA lawsuits with triple damages.
Ya know, on the one hand, I wonder how many federal officials own nVidia stock and stock of other AI companies...
But then again, on the other, I wonder how many nVidia employees (and those of other AI companies!) own government T-bills (and other government backed securities!) in their 401K or other investment accounts!
You know, everybody in large corporations and governments owning shares in other large corporations and governments -- adds a whole new level of meaning to "stakeholder capitalism"... :-)
(In the far future, there'll probably be small foreign governments which become their own ticker symbol -- and get publicly traded on various stock exchanges! :-) You know, like: "Would you like to buy a share in Denmark (DNMK) or perhaps Norway (NRWY)? I hear they're very competitive with Apple (AAPL) right now! Great price toearnings (PE) ratio!" :-) )
I am simplifying terribly but just as shares represent a fraction of a company, cash represents a share of a nation’s economy.