I wanted to use proc http to scrape quotes off Yahoo finance. It did not produce the HTML in the out file, but when I used debug level = 3 to figure what happened, the full HTML was given in the log. What happened? Clearly, I want the HTML in the out file, but the alternative of saving the log as a text will be sufficient as well. How can I do that?
filename Testing "&Folder&OutFile";
%let YahooFin = "";
proc http
url = &YahooFin.
out = Testing
method = "get";
debug level = 3;
run;
EDIT
Here are my macro variables. Just to clarify, I did receive a HTML from my original script, but it was just the top portion, not the whole thing. See the enclosed image.
%let Folder = /06specialty/Practice Scripts/;
%let OutFile = TestSNAP.txt;
3 Answers
The HTML that link generates has some really long lines (541,567 bytes).
Make sure to tell SAS to use a long enough LRECL if you are going to try to read it with a DATA step.
1149 proc http
1150 url = &YahooFin.
1151 out = Testing
1152 method = "get";
1153 run;
NOTE: PROCEDURE HTTP used (Total process time):
real time 0.98 seconds
cpu time 0.04 seconds
NOTE: 200 OK
1154
1155 data _null_;
1156 infile testing lrecl=1000000 length=ll ;
1157 input;
1158 run;
NOTE: The infile TESTING is:
Filename=XXXX\#LN00123,
RECFM=V,LRECL=1000000,File Size (bytes)=777076,
Last Modified=21Mar2021:15:23:01,
Create Time=21Mar2021:15:20:45
NOTE: 116 records were read from the infile TESTING.
The minimum record length was 0.
The maximum record length was 541567.
NOTE: DATA statement used (Total process time):
What does your OutFile macro-variable resolve to? What is the extension of your output file?
I tried with a .txt file:
filename Testing "/path/to/want.txt";
%let YahooFin = "";
proc http
url = &YahooFin.
out = Testing
method = "get";
run;
And it worked:
I would suggest to use Yahoo Graph API to get quotes or historical price for a stock
/*Lets fetch Historic data from Microsoft*/
%let stock=MSFT;
filename test "%sysfunc(getoption(WORK))\&stock..json";
proc http
url="(&)interval=1d%str(&)indicators=quote%str(&)includeTimestamps=true%str(&)includeTimestamps=true%str(&)crumb=&getCrumb."
out=test;
*debug level=3;
quit;
LIBNAME stock JSON "%sysfunc(getoption(WORK))\&stock..json";
You can get more details from
