Scraping Web Page Data Automatically with Excel VBA

We learnt earlier how to scrape web data from web pages with Excel VBA using the inbuilt features of MS Excel and also from which had a great html structure and allowed us to extract data quickly and easily. But how do you get data from a web page which has a difficult form with a button that has no id or name property? You need to adapt your VBA code to do the job. So we use the ‘form submit’ property and the ‘td’ elements of a table with a ‘for loop’ to make the data…

36 thoughts on “Scraping Web Page Data Automatically with Excel VBA

  1. Dear Dinesh Sir,
    My requirement is slightly different. Actually we have a team located globally and we maintain schedule for releases we manage individually on excel on our laptops. Now we have a sharepoint where in we have created 1 document with combined schedules. It is difficult to everytime open and update any change manually on share point.. Thus, is it possible to update the sharepoint excel sheet with latest updates that are made on schedule in excel on laptop with a click of a button? Please advice.
    Best regards,

  2. Sir,

    Is it possible log into my gmail account to get data to my excel, Data having a particular Subject, or a particular sender , that is be searched using input box.

    any video over this then please share.

  3. Thank you very much Dinesh  . The video was helpful .Could you please take a separate session on how to pick the elements to code ie for elements that don't have name or id  or could you please suggest any link that could provide such information ? That could be of great help to understand browser automation . I am a beginner in excel macros and have learnt to write codes that perform excel activities only . On looking a your video I was able to understand and relate to what was being done .But it would be great if you could show the page source as and when you code . This could help understand on how the elements are picked appropriately .Thanks Again!!

  4. Hi Sir,

    I am a regular follower of your channel. These videos are really amazing and I got knowledge. Thanks.

    Now I have a ques..Can you please provide a video on how to extract web page data bt chrome instead of internet explorer? It will be helpful.

  5. Hi Sir while practicing your code i am getting stuck at this code line form(0).submit and showing me an automation error.

    I dont know what am i doing wrong. Currently i am using excel 2013, i hope thats not the cause for this. Any help on this is appreciated as i am very keen on learning web scrapping tutorials given by you

  6. Hello can anyone help on this this, a web page having multiple tables (5 nos.). required table which needed in excel sheet fall under form name historicalForm as mentioned below, pls suggest thank in advance.

    <form name="historicalForm" onsubmit="return false;">
    <div class="tabular-data-historic" id="historicalData">

  7. Hi,
    in continuation of previous post….
    first problem am facing is Multiple combobox in form, which dropdown options are linked, if we change the first combobox item, then next combobox options change as well . pls suggust…….

  8. Hi,
    great videos, but i have a problem to get usefull thing from

    in my code:

    ie.document.getElementsbyName("string").Item.innertext = szukana_rzecz
    ie.document.getElementsbyName("search_scope").Item(0).Value = typ_kategori

    I don't know how to get a Button for search and choce dropdown list. How to do that?

  9. Hi Sir,
    I should thank you sir, i saw your videos every day from there i am getting lot knowledge, I have a big requirement that is i need to scarp data from zoom info website if put web address in web address box it will open automatically and give it that address , city, state, zip code, phone number, fax number into excel sheet automatically i requesting you please do this requirement , it will helpful for all your viewers. thank you sir

  10. this is great! Thanks for sharing Mr. Kumar. I want to ask one thing in regards to the Data extraction from website. I have this duty to extract the rate from every day for certain periods of time. Do u think u will be able to show us what need to do?

  11. Hi great channel, do you have a video showing the following.
    I have a spreadsheet with a list of codes, which when entered into a website, show information I want pasting next to this code, if possible I would like to automate this process as there could be thousands. Thanks in advance

  12. Hello Mr.Takyar
    Very helpful your video.. but im still struggling to find a way to scrape from a range of data from web to vba… I watch your videos but still couldnt find the best way to do that..
    eg. I have 2 textbox.. and I need put an interval of values there.. like: search (X1) to (X2) , and bring me a table of values between those intervals, those values are in a database in a website.
    Thank you

  13. Hello Sir,

    I follow your videos and have been learning a lot from your videos, Thank You very much for shearing your knowledge and experience.
    I am working on a project that requires "Scraping a Web Page Data Automatically" and I have run into a problem that is driving me crazy (sleepless nights). My problem is that when i access the source code of the page and Loop through each ("div") elementByTagName and return the InnerHTML the value returned is limited in scope. The best way to phrase it is that it return the InnerHTML but the InnerHTML has another layer of information that is not being disclosed. However, If i was to use "FireFox" and Inspect Element and rightclick the element and copy InnerHTML then ALL of the innerHTML and layers of data will be available. If you can assist me or direct me to information on a solution for this problem that would really be helpful.

    Thank you.

  14. I really appreciate your hard work on making VBA videos. I'm one of your students through these videos.
    Well, I am having some serious troubles. I am trying to log in to my "router" through a vba code.
    So far through vba,
    I can open IE (even though I prefer Chrome or Mozille)
    I can enter username
    I can enter password
    but I cannot click on "submit".
    This is the only problem.

    Here is the source code of the router log in page…

    <td align="center" width="128" height="32"> <a href="javascript:Apply();" class="appmsg" onMouseOut="MM_swapImgRestore()" onMouseOver="MM_swapImage('Image1','','../../images/create_p.png',1)"><img src="../../images/create_n.png" alt="Create" name="Image1" width="124" height="38" border="0"></a>

  15. Hi Sir , I have similar kind of doubt for google keyword planner.
    I want to run VBA code that login to google keyword planner, and get keyword search volume by device, like computer volume, mobile volume, tablet volume. Can we get that data using VBA?

  16. but what if the site uses Google search and targets by for example geographic region as a default i.e. all of California.  Then type in keyword or phrase you want to search by and the results are for example q=keyword and cx=appears to be a variable.  What do you do in this case?  clicking the button for the search results yields the narrowed search results.  But you want the cleaned up title, description and cached url to dump in a spreadsheet and you have to do this 30+ times on govt websites to gather information.  So what do you do?  I followed your former video word-for-word, however, drastically changed their website.  I'm only a beginner though I do know HTML. …ps and thank you for your videos btw

  17. Very good video, good work. 
    May I know what is the best programming language to do web data scraping please?
    I wanted to scrape the program names, fees, duration from multiple college websites. can you please advise, since there are thousands of colleges to pull the data from so I am concerned to use the good performance programming. Actually i was thinking to use php-curl.

  18. Hello. Thanks for this example. I'm trying to do something similar with and I'm getting this error. In your code you have 'Set button = form(0).onsubmit' When I run this I get a debug error of: run-time error '13': Type mismatch on the above mentioned code. Do you know why I'm getting this when I'm trying to implement this on site?

  19. Hi,
    First of all thanks for your help. but it would be highly appreciated if you help solve my problem.
    In my situation I have come across pop up calendar to be filled on a website.
    Can you please explain how to write down the code for that?
    Thanks once again.

  20. One issue I have is that in secure sites or sites that need log-ins web query fail in a spectacular fashion as the cookies don't update and thus scrapes the wrong information.

  21. Really cool, but is there any way to extract data from a webpage using excel that doesn't have tables? I'm trying to extract data from description on a webpage and maybe has some features listed with bullets. Can excel do this type of extracting or is it limited to tables? I'd like to define how excel can extract by say first couple of words or first word in the description if possible. An also to define exactly which cells get the data I'm extracting from a webpage. Thanks for the insight on how this could be done. The way you explain code is very easy to understand.

Leave a Reply

Your email address will not be published. Required fields are marked *