如何更改url package通路HTTP時的user-agent header

原文位址:https://www.lujun9972.win/blog/2021/09/24/如何更改url-package通路http時的user-agent-header/index.html

有些網站會根據 http request 中的 user-agent header 的值傳回不同的response，例如 http://wttr.in 會根據就會根據 user-agent 是否為 curl 來決定是傳回帶圖檔的HTML，還是字元拼接圖案的文本。

一開始我以為修改

url package

中的

user-agent

就是直接把相應的 header 内容加到

url-request-extra-headers

中就行了，事實證明我還是太天真了，這樣做的後果是會産生兩個

user-agent

header...

(let ((url-debug t)
      (url-request-extra-headers '(("User-Agent" . "curl/7.78.0"))))
  (kill-buffer (url-retrieve-synchronously "http://wttr.in"))
  (with-current-buffer "*URL-DEBUG*"
    (keep-lines "^User-Agent" (point-min) (point-max))
    (buffer-substring-no-properties (point-min) (point-max))))
      

User-Agent: URL/Emacs Emacs/28.0.50 (X11; x86_64-pc-linux-gnu)
User-Agent: curl/7.78.0
      

在翻閱了 url manual 之後才知道，原來

url

專門有個變量用來控制 user-agent:

url-user-agent is a variable defined in ‘url-vars.el’.

Its value is ‘default’

  You can customize this variable.
  This variable was introduced, or its default value was changed, in
  version 26.1 of Emacs.
  Probably introduced at or before Emacs version 25.1.

User Agent used by the URL package for HTTP/HTTPS requests.
Should be one of:
* A string (not including the "User-Agent:" prefix)
* A function of no arguments, returning a string
* ‘default’ (to compute a value according to ‘url-privacy-level’)
* nil (to omit the User-Agent header entirely)
      

是以修改

user-agent

header 的正确方法是修改

url-user-agent

這個變量的值:

(let ((url-debug t)
      (url-user-agent "curl/7.78.0"))
  (kill-buffer (url-retrieve-synchronously "http://wttr.in"))
  (with-current-buffer "*URL-DEBUG*"
    (keep-lines "^User-Agent" (point-min) (point-max))
    (buffer-substring-no-properties (point-min) (point-max))))
      

User-Agent: curl/7.78.0