Excel download file from url

Как скачать файл из интернета по ссылке

Вся суть статьи уже в заголовке. Возникает порой необходимость скачивания файлов из интернета только на основании ссылки. Например, это какие-то постоянно меняющиеся данные или автоматически генерируемая другим кодом ссылка. Или еще более усугубленный вариант — строк 100 со ссылками на файлы, которые надо скачать. Вот уж радости руками по каждой клацать 🙂
Поэтому выкладываю решение, которое в большинстве случае поможет при помощи Visual Basic for Applications скачать файл на основании ссылки URL:

‘————————————————————————————— ‘ File : mDownloadFileFromURL ‘ Purpose: код позволяет скачивать файлы из интернета по указанной ссылке ‘————————————————————————————— Option Explicit ‘объявление функции API — URLDownloadToFile ‘ работает на любых ПК под управлением ОС Windows ‘ на MAC код работать не будет #If Win64 Then ‘для операционных систем с 64-разрядной архитектурой Declare PtrSafe Function URLDownloadToFile Lib «urlmon» Alias «URLDownloadToFileA» _ (ByVal pCaller As LongLong, ByVal szURL As String, ByVal szFileName As String, _ ByVal dwReserved As LongLong, ByVal lpfnCB As LongLong) As LongLong #Else #If VBA7 Then ‘для любых операционных систем с офисом 2010 и выше Declare PtrSafe Function URLDownloadToFile Lib «urlmon» Alias «URLDownloadToFileA» _ (ByVal pCaller As LongPtr, ByVal szURL As String, ByVal szFileName As String, _ ByVal dwReserved As Long, ByVal lpfnCB As LongPtr) As LongPtr #Else ‘для 32-разрядных операционных систем Declare Function URLDownloadToFile Lib «urlmon» Alias «URLDownloadToFileA» _ (ByVal pCaller As Long, ByVal szURL As String, ByVal szFileName As String, _ ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long #End If #End If ‘переменная для хранения пути к папке Dim sFilePath As String Function CallDownload(sFileURL As String, sFileName As String) ‘ sFileURL — ссылка URL для скачивания файла ‘ sFileName — имя файла с расширением, которое будет присвоено после скачивания Dim h If sFilePath = «» Then ‘диалоговое окно выбора папки ‘подробнее: http://www.excel-vba.ru/chto-umeet-excel/dialogovoe-okno-vybora-fajlovpapki/ With Application.FileDialog(msoFileDialogFolderPicker) If .Show = False Then Exit Function End If sFilePath = .SelectedItems(1) End With End If If Right(sFilePath, 1) <> «» Then sFilePath = sFilePath & «» ‘проверяем есть ли файл с таким же именем в выбранной папке If Dir(sFilePath & sFileName, 16) = «» Then ‘файла нет — скачиваем h = DownloadFileAPI(sFileURL, sFilePath & sFileName) Else ‘файл есть — запрос на перезапись If MsgBox(«Этот файл уже существует в папке: » & sFilePath & vbNewLine & «Перезаписать?», vbYesNo, «www.excel-vba.ru») = vbYes Then ‘если существующий файл открыт — невозможно его перезаписать, показываем инф.окно ‘отменяем загрузку If IsBookOpen(sFileName) Then MsgBox «Невозможно сохранить файл в указанную папку, т.к. она уже содержит файл ‘» & sFileName & «‘ и этот файл открыт.» & _ vbNewLine & «Закройте открытый файл и повторите попытку.», vbCritical, «www.excel-vba.ru» Else h = DownloadFileAPI(sFileURL, sFilePath & sFileName) End If End If End If CallDownload = h End Function ‘функция скачивания файла в выбранную папку Function DownloadFileAPI(sFileURL, ToPathName) ‘ sFileURL — ссылка URL для скачивания файла ‘ ToPathName — полный путь с именем файла для сохранения Dim h Dim sFilePath As String Dim sFileName As String ‘вызов функции API для непосредственно скачивания h = (URLDownloadToFile(0, sFileURL, ToPathName, 0, 0) = 0) ‘если h = False — файл не удалось скачать, показываем инф.окно If h = False Then MsgBox «Невозможно скачать файл.» & vbNewLine & _ «Возможно, у Вас нет прав на создание файлов в выбранной директории.» & vbNewLine & _ «Попробуйте выбрать другую папку для сохранения», vbInformation, «www.excel-vba.ru» Exit Function Else ‘файл успешно скачан sFileName = Dir(ToPathName, 16) sFilePath = Replace(ToPathName, sFileName, «») If MsgBox(«Файл сохранен в папку: » & sFilePath & _ vbNewLine & «Открыть файл сейчас?», vbYesNo, «www.excel-vba.ru») = vbYes Then If IsBookOpen(sFileName) Then MsgBox «Файл с именем ‘» & sFileName & «‘ уже открыт. Закройте открытый файл и повторите попытку.», vbCritical, «www.excel-vba.ru» Else Workbooks.Open ToPathName End If End If End If DownloadFileAPI = h End Function ‘Функция проверки — открыта ли книга с заданным именем ‘подробнее: ‘ http://www.excel-vba.ru/chto-umeet-excel/kak-proverit-otkryta-li-kniga/ Function IsBookOpen(wbName As String) As Boolean Dim wbBook As Workbook For Each wbBook In Workbooks If Windows(wbBook.Name).Visible Then If wbBook.Name = wbName Then IsBookOpen = True: Exit For End If Next wbBook End Function

Код необходимо скопировать и вставить в книгу в стандартный модуль. Макросы должны быть разрешены.

Основная функция , отвечающая за непосредственно скачивание — это функция API(Application Programming Interface) URLDownloadToFile . Она объявлена в самом верху кода. Там есть страшные директивы вроде #If Win64 Then . Это особые директивы, которые работают даже вне процедур. Поэтому не надо удивляться, что они вне всяких Sub и тем более не надо эти Sub-ы добавлять. При этом так же не надо удивляться, если какие-то из строк внутри этих директив будут подсвечены компилятором VBA красным шрифтом. На функциональность это не повлияет.

Вызов скачивания файла происходит обычным обращением к функции CallDownload. Например, есть ссылка для скачивания: http://www.excel-vba.ru/files/book.xls . И сохранить надо под именем » Книга1.xls «. Вызываем функцию скачивания файла:

Sub DownloadFile() Call CallDownload(«http://www.excel-vba.ru/files/book.xls», «Книга1.xls») ‘вызываем скачивание файла End Sub

Функция сама запросит папку для сохранения файла и после скачивания предложит открыть этот файл. Если такой файл уже есть — будет предложено его перезаписать.
К статье приложен файл, в котором код чуть расширен — он позволяет скачивать файлы сразу из множества ячеек, проставляя при этом признак — скачан файл или нет. И если сразу весь список обработать не получилось и какие-то файлы остались не скачанные(например, имена совпадали, а заменять файлы не надо было), то в этом случае можно будет повторно запустить код и скачиваться будут лишь те, у которых статус не является » Скачан! «.
Так же т.к. ячеек много, перед скачиванием файлов будет выбор — запрашивать ли открытие файлов после скачивания или нет. Если открывать не надо, следует ответить Нет. Тогда файлы просто будут скачаны в указанную папку. Однако, если в этой папке будут расположены файлы с идентичными именами — запрос на перезапись все же появится, при этом для каждого файла. Если подобный запрос так же мешает, то надо этот блок:

‘проверяем есть ли файл с таким же именем в выбранной папке If Dir(sFilePath & sFileName, 16) = «» Then ‘файла нет — скачиваем h = DownloadFileAPI(sFileURL, sFilePath & sFileName) Else ‘файл есть — запрос на перезапись If MsgBox(«Этот файл уже существует в папке: » & sFilePath & vbNewLine & «Перезаписать?», vbYesNo, «www.excel-vba.ru») = vbYes Then ‘если существующий файл открыт — невозможно его перезаписать, показываем инф.окно ‘отменяем загрузку If IsBookOpen(sFileName) Then MsgBox «Невозможно сохранить файл в указанную папку, т.к. она уже содержит файл ‘» & sFileName & «‘ и этот файл открыт.» & _ vbNewLine & «Закройте открытый файл и повторите попытку.», vbCritical, «www.excel-vba.ru» Else h = DownloadFileAPI(sFileURL, sFilePath & sFileName) End If End If End If

заменить на всего одну строку:

h = DownloadFileAPI(sFileURL, sFilePath & sFileName)

Но при этом надо помнить — что при этом можно потерять какие-то важные файлы. Поэтому подобные вещи вы делаете на свой страх и риск.

Однако следует помнить одну вещь: не все сайты вот так запросто разрешают скачивать с них файлы, тем более пачками. Особенно это актуально для всякого рода форексов и иже с ними. Возможно, получится скачать один, два, три — десять файлов. Но всегда может случиться так, что сайт просто заблокирует ваш IP до конца дня, т.к. на сайте установлено ограничение на автоматизированное обращение извне. При этом для разных сайтов решение данной проблемы может быть различным и не всегда решаемым

Tips_Macro_DownloadFileFromURL.xls (64,0 KiB, 3 799 скачиваний)

Статья помогла? Поделись ссылкой с друзьями!

Источник

Excel-VBA Solutions

Want to become an expert in VBA? So this is the right place for you. This blog mainly focus on teaching how to apply Visual Basic for Microsoft Excel. So improve the functionality of your excel workbooks with the aid of this blog. Also ask any questions you have regarding MS Excel and applying VBA. We are happy to assist you.

Pages

Download a file from url using VBA

Sometimes our Excel VBA applications need to interact with websites. Downloading a file through a URL is a typical example. In this lesson you can learn how to do that using XMLHttpRequest and ADODB.Stream object. XMLHttp is used to request the data from the web server. Once we receive the data from the server, the ADODB.Stream object is used to write that data to a file. You can use this method to download file types such as image files, csv files etc.

So let’s start writing our macro to download a file using the Uniform Resource Locator (URL). Let’s name our Sub procedure as “DownloadFileFromURL”

First we need to declare a few variables.

Next assign the URL of the file to the FileUrl variables

For an example if your file URL is https://www.excelvbasolutions.com/images/chart.jpg then you can assign it to the variable as follows.

Next step is to create a XMLHttp object to request the data from the server.

Call the open method followed by the send method.

Then check the status. If it equals 200 (OK), we can move to the next step.

Now we need to create an ADODB.Stream object to write the data(received) to a file.

Stream in programming means flow of data. In programming data is what flows between processors and input or output devices. Check this link to see all the Properties, Methods and Events of Stream object.

But we need to use only a few of them here.

In the SaveToFile method I have entered 2 as SaveOptions. So the program will overwrite the file with the data from the stream if the file already exists. If you don’t want to overwrite the existing file then replace it with 1.

Below is the full code which you can use to download a file through a URL.

Dim FileUrl As String
Dim objXmlHttpReq As Object
Dim objStream As Object

FileUrl = «URL of your file»

‘example
‘FileUrl = «https://www.excelvbasolutions.com/images/chart.jpg»

Set objXmlHttpReq = CreateObject(«Microsoft.XMLHTTP»)
objXmlHttpReq.Open «GET», FileUrl, False, «username», «password»
objXmlHttpReq.send

If objXmlHttpReq.Status = 200 Then
Set objStream = CreateObject(«ADODB.Stream»)
objStream.Open
objStream.Type = 1
objStream.Write objXmlHttpReq.responseBody
objStream.SaveToFile ThisWorkbook.Path & «» & «file.jpg», 2
objStream.Close
End If

In the above example I showed how to download a .jpg file. But you can use this method to download other file types as well. Remember to change the file extension depending on the type of the file you download. For example if you are downloading a CSV file then the SaveToFile method should be changed as follows.

Источник

Download Files with VBA URLDownloadToFile

The VBA Tutorials Blog

Automation is all the rage these days. Automation can thrash labor markets while simultaneously exploding productivity and profits. It will also make human jobs less boring (assuming we still have jobs). In this tutorial, we will learn how to use VBA to programmatically download files based on URLs. We might already know these URLs, or we may have to scrape them from the web and parse them. This is automation of a rather dull task, so implementing it hopefully has a positive impact on your work.

This tutorial is a little more involved than our recent tutorials. It contains multiple steps and pieces of code that are outside the general scope of a “downloading files with VBA” tutorial. Where it makes sense to explain, we’ll explain, but there are times when we’ll simply gloss over the technical parts and just use the code without much explanation. This guide lays out the practical requirements for downloading files from the internet using VBA. Once you finish the tutorial, you’ll be able to easily download URL files yourself.

With that said, let’s get started.

The Built-in Windows Library Function

There are a number ways to download files using VBA, including using an XMLHTTP request to stream the binary data or text associated with your file, and then using the SaveToFile method to finish saving the downloaded file. This is probably the most common way to download files, but we’re not going to do it this way.

In this tutorial, we’re going to download files using a Windows function called URLDownloadToFile . The URLDownloadToFile function is part of the Windows library urlmon. Before we can use it, we need to declare the function and connect to it from VBA. We can do that by placing this line of code at the top of a VBA module:

Make powerful macros with our free VBA Developer Kit

It’s easy to copy and paste a macro like this, but it’s harder make one on your own. To help you make macros like this, we built a free VBA Developer Kit and wrote the Big Book of Excel VBA Macros full of hundreds of pre-built macros to help you master file I/O, arrays, strings and more — grab your free copy below.

I know this is a rather long function declaration. Libraries are basically sets of prewritten code, usually optimized by the library developer (Microsoft, for this library). All the code needed to access a URL, download the stream of bits and bytes, then structure them back into a file on the hard drive is conveniently wrapped into this single function. All we have to do is throw this declaration at the top to begin using it.

We can see the return type is Long , which means the function will return a whole number. This makes checking whether the download started much easier. We’ll get to that soon, but for now, the most important parts are the szURL and szFileName parameters. These arguments represent the URL source and the name of the file after we’ve downloaded it. The other arguments can almost always be set to 0 .

Downloading Our First File using VBA

Let’s say you find a picture you like and you want to download it. You could right-click then “Save Image as…”, or you could use VBA. Admittedly, using VBA would be inane for a single picture, but if you have hundreds of pictures to download, this task would be a perfect automation target. This tutorial focuses on downloading images, but the function is just as useful for downloading other file types.

Anyway, we’re going to start with just one picture: a cityscape picture from Hong Kong’s Wikipedia page as of November 2018. In the HTML, every picture on every webpage is represented as a link to the picture’s very own URL, and that’s the URL we need to pass to the URLDownloadToFile function. You do not want to pass the URL of the webpage where the picture is embedded.

Under the architecture section on the Hong Kong Wikipedia page, there is a nice panorama of Hong Kong at night. We can access its URL by right-clicking and then clicking “Copy Image Location” or “Copy Image Address.” The words may change depending on your browser, but you get the idea.

Copy the URL of the image by clicking the «Copy Image Location» option.

You should have the following image path on your clipboard:

. If you paste this path directly into a web browser’s URL box, you’ll get the image on its own page.

To download the picture with VBA, we can run a short subroutine, like this:

If you have a folder named C:DownloadedPics , there will now be a file entitled HK Skyline.jpg in that folder. Simple as that. Notice how we concatenated our download path ( dlpath ) and our file name. Because we do that, it’s important to have a trailing backslash at the end of your download path.

If a folder with the path you specify doesn’t exist, your download will fail. You won’t get any error messages, but your file won’t be downloaded anywhere.

If the download folder doesn’t exist, don’t worry. You can make the folder using VBA before calling the URLDownloadToFile function. This approach is particularly useful if you frequently automate repetitive tasks. For example, you can timestamp the folder name when you run the program so you can go back through the folders later and know when you ran the macro.

Downloading Multiple Files

If you’re going to download multiple files, you likely already have a list of URLs. Lists are nice, because you can stick them in an array then iterate through your array with a For Loop. You just need to program a way to change the downloaded location’s filepath each time, otherwise you’ll overwrite the previous download with each new file. It’s also a smart idea to check for delays when downloading multiple files. We’ll explain why momentarily.

Designing with Uniqueness in Mind

If the list of files you want to download is small enough, you could manually label each URL and use the label as the filename. For example, you could have 2 columns in Excel: the first column will contain the file name and the second column will contain the URL you want to download, like this:

A list of cities (labels) and a URL with a photo of the respective city

Now we can iterate through this list, downloading each picture and assigning it a unique name.

Since we know we only want to download jpg pictures in this example, we hard-coded the .jpg extension in our URLDownloadToFile function. Once downloaded, we should have a folder with files looking something like this:

The destination folder with six downloaded pictures of cities

Timeout Code to Track Problems

If you download several hundred files per run, you will want to track which files are missing. An easy way to implement a timeout is to check the URLDownloadToFile return value and wait a couple seconds if it is not the desired result. If it hangs for more than the specified time, move to the next URL and mark that file’s download as failed.

A return value of zero ( 0 ) means the download has started successfully. Barring some connection interruption or memory failure downloading a gigantic file, it should finish. Thus, a simple yet effective way to track download problems is to check whether the result of the function is zero. If it isn’t, try it one more time before moving on. Other return values are out-of-memory or unreachable-source errors. If you want to be very meticulous, you could mark the type of failure in your program (hint: use an array).

For a more robust solution, you can use VBA to check whether or not the files exist before exiting your macro. If they don’t, you know the download failed or it’s still downloading.

In this example, we capture the returning long integer and check whether or not it’s a zero. If it’s not, we force the macro to try it again. If it fails a second time, we just give up and move on. We don’t implement a way to flag it as failed in this macro, but I recommend you store details about the failed download into an array before moving on.

Scraping for File Locations

If you want to grab every picture on a website, you can capture the URLs of the images via webscraping.

Since images will be tagged with img in the webpage’s HTML, you can use VBA’s GetElementsByTagName function to grab all the image URLs. You can easily adapt our VBA webscraping tutorial to grab the img tag. You will then need to access the src property of each Item, which will contain the URL of the image file. If the src property is not available for some reason, you might be able to use the href property. At least one of these two properties will be present for all images.

The modification from the webscraping tutorial would follow logic like this:

The example we used for webscraping used the Wikipedia page on country and dependency populations, which has a nice table of countries with ranks, populations, and little flags. To practice, try to scrape the table, extract the country name, scrape the associated flag’s image URL, download the flag picture file, and use the extracted country name to name the file. Paste your solution in the comments section!

If you aren’t so ambitious, you could use something like imgname = i & downloadPath , where i is the for-loop control variable. This method would match the flag with the country’s population rank, so there is some practicality to doing it this simpler way.

If you’re lucky, the photos on the website might have a title property, and you could use that for naming, too.

Ethical Questions

You can automate the download of hundreds or thousands of files rather easily using this method, especially when coupled with a webscraper that’s simply looking for a tag, like the img tag, to grab any and all files on a website.

There are three ethical issues here.

First, you’ll be requesting server time from someone or some company, and automating this process might put strain on their infrastructure. In the extrinsic case, you might get your IP banned from accessing the server as a punishment, but even if you don’t, there’s the ethical question of using up resources others are providing for free and fair use. For huge sites, like Google or Amazon, they probably won’t notice at all. But smaller companies, especially those running their own servers, might be affected.

The second ethical question revolves around hotlinking and revenue theft. Basically, if you already know the image URL and don’t need to visit the main webpage (such as if you found the image through Google’s Image Search and just grabbed the URL), you’ll be depriving the host of any advertising revenue. On one hand, you won’t be taking up server resources to load the main page, but you also won’t be rewarding the host with any advertising revenue or traffic in exchange for their information. This could harm their search engine rankings.

While this kind of automation probably doesn’t carry ethical questions of eliminating someone’s job, it does carry some ethical implications. You should think about such ethical considerations before automating any task.

The third ethical question, and the one that plagued me the most before posting this tutorial, is that someone with criminal intent could use the function for more sinister macros. For example, someone could write a macro to automatically download viruses or other nefarious file types to someone’s machine. I’m posting this tutorial because many people would benefit from being able to automatically download files using VBA. Don’t be a jerk and abuse the knowledge you’ve learned as a VBA developer.

Conclusion

In this tutorial, you learned how to use VBA to download files. We used images in our examples, but you can download any file type. The entire automation process can be quite long and may require a lot of research, but this tutorial will get you on your way. Automation can help reduce tedious work, but always remember to consider the ethical implications that arise from automating work, from job elimination to revenue theft.

I hope you found this helpful. When you’re ready to take your VBA to the next level, remember to subscribe using the form below.

Ready to do more with VBA?
We put together a giant PDF with over 300 pre-built macros and we want you to have it for free. Enter your email address below and we’ll send you a copy along with our VBA Developer Kit, loaded with VBA tips, tricks and shortcuts.

Before we go, I want to let you know we designed a suite of VBA Cheat Sheets to make it easier for you to write better macros. We included over 200 tips and 140 macro examples so they have everything you need to know to become a better VBA programmer.

This article was written by Cory Sarver, a contributing writer for The VBA Tutorials Blog. Visit him on LinkedIn and his personal page.

Источник