Follow us on Twitter!
I'd prefer to die standing, than to live on my knees - Che Guevara
Wednesday, April 23, 2014
Navigation
Home
HellBoundHackers Main:
HellBoundHackers Find:
HellBoundHackers Information:
Learn
Communicate
Submit
Shop
Challenges
HellBoundHackers Exploit:
HellBoundHackers Programming:
HellBoundHackers Think:
HellBoundHackers Track:
HellBoundHackers Patch:
HellBoundHackers Other:
HellBoundHackers Need Help?
Other
Members Online
Total Online: 20
Guests Online: 18
Members Online: 2

Registered Members: 82886
Newest Member: The Slummy
Latest Articles
View Thread

HellBound Hackers | Computer General | Programming

Author

Downloading from a https webserver


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 01:16
I am attempting to download a zip file automatically that has data on potatos and the like from a https webserver, I have a username and password, but I want to be able to download and save automatically, since this program will be running as a service. The site does not use cookies to authenticate and it does not have port 21 open, the only port open on the server is 443. My current solution is to use a third party macro tool that will interact with my program to click the save button and then type in a file name to save it as.

I am using MS C# 2010 and have been using the webbrowser control, simply because I do not know another method. I have tried going to the download page without authenticating using the webclient class, and it just redirects me to the default.aspx page.I have tried using a webclient class and using the network credentials property to authenticate before downloading, but still no luck. Any ideas?

Thanks,

wired_al
Author

RE: Downloading from a https webserver

spyware
Member



Posts: 4192
Location:
Joined: 14.04.07
Rank:
God
Warn Level: 90
Posted on 11-10-10 01:19
cron + wget in linux, scheduler + wget in windows.



img507.imageshack.us/img507/3580/spynewsig3il1.png
"The chowner of property." - Zeph
[small]
Widespread intellectual and moral docility may be convenient for leaders in the short term,
but it is suicidal for nations in the long term.
- Carl Sagan
[center]�Since the grid is inescapable, what were the earlier lasers about? Does the corridor have a sense of humor?� - Ebert[/ce
Author

RE: Downloading from a https webserver


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 04:23
The only problem is something I should have probably stated earlier... I do not have access to the location of the file in a way that I know how to get to it... it's all done through a combination of javascript and ASPX. I just got the source from the downall page... which I will post. Basically, I know they are running IIS, wget wont work, because it does not like unauthenticated download attempts...

Here is the code...

Code


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head><title>
   Download All Files
</title></head>
<body>
    <form name="form1" method="post" action="downloadAll.aspx?userName=xxxxxxxxxx" id="form1">
<div>
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/nothing/important"> <!--note, edited to protect clients data -->
</div>

    <div>
    <span id="thisLabel"></span>
    </div>
    </form>

<script language='javascript'>alert( "Could not find file 'C:\inetpub\TempUser\xxxxxxxxxxFFVFiles.zip'." );</script>
</body>
</html>





It is a state run government website, and we are trying to talk to the admin, he won't even reply... government workers...
Also, this service is not only a download service, it also extracts the data from the xml files which are in the zip file, then parses the xml files, stores them in the on site SQL database, and then pulls reports. All of this is working except for the download...
Author

RE: Downloading from a https webserver

stealth-
Member



Posts: 1003
Location: Eh?
Joined: 10.04.09
Rank:
Mad User
Posted on 11-10-10 04:41
wired_al wrote:
wget wont work, because it does not like unauthenticated download attempts...


Wget can handle authentication.


wget --user=fbi --password=s3cr3t http://site.gov/downloadAll.aspx?userName=xxxxxxxxxx


Something like that?
It's a little hard to tell considering you erased over half the HTML.


The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
http://www.stealt. . .

Edited by stealth- on 11-10-10 04:42
http://www.stealth-x.com
Author

RE: Downloading from a https webserver


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 04:56
stealth- wrote:
wired_al wrote:
wget wont work, because it does not like unauthenticated download attempts...


Wget can handle authentication.


wget --user=fbi --password=s3cr3t http://site.gov/downloadAll.aspx?userName=xxxxxxxxxx


Something like that?
It's a little hard to tell considering you erased over half the HTML.


Sadly, I did not erase the html... I changed the username, and I erased one hidden field that had some base64 encryped data in it. I decoded it and it is not relevant...

I just tried the code for wget that you posted (except of course I changed the username password and url) and it gave me the source to the main page because it didn't like my authentication...

Thanks
-wired_al
Author

RE: Downloading from a https webserver

stealth-
Member



Posts: 1003
Location: Eh?
Joined: 10.04.09
Rank:
Mad User
Posted on 11-10-10 05:10
1. How does this server authenticate with the user again?
2. You remembered it's https://, right?
3. The HTML code you posted is not a functioning page, so obviously something is wrong.


The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
http://www.stealt. . .
http://www.stealth-x.com
Author

RE: Downloading from a https webserver


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 06:13
stealth- wrote:
1. How does this server authenticate with the user again?
2. You remembered it's https://, right?
3. The HTML code you posted is not a functioning page, so obviously something is wrong.


1) A username and password prompt at the default.aspx (cookies are not used, and I cannot find a session ID either...)
2) yes
3) Probably because it is a .aspx page that is never actually seen. I had to type view-source: in firefox in front of the address. This is what happens when you click the download all link. It brings up a popup window which is the downAll.aspx?userName=xxxxxxxxxx , and then starts the download of the zip file and closes the little popup. This page is never ment to be seen by the user I don't think. (note: aspx code is run at the server. kind of like php, only the output is sent to the browser) When I click the date link to choose what data I want, this function is executed

Code

function showFiles(user, fileDate){
   //setValue("")
   document.body.style.cursor = 'wait';
   PageMethods.getFilesByDate(user, fileDate, OnSucess, OnFail);
   var url = "javascript:downAll('" + user + "');";
   $get('ctl00_centerBox_downAllFiles').innerHTML = "<a class='a' href=" + url + ">Download All Files (zip file)</a>";
}





The user is my username, and the fileData is the date I want data for.
When I click the download all button, this code is executed.
Code

function downAll(user){
      window.open("downloadAll.aspx?userName=" + user,"", "height=2,width=2");
   }




which results in the code I posted earlier.

When I tell it to only download one file, this is the funtion that is executed
Code

function checkForExpiredSession(linkURL) {
    //setValue("")
    PageMethods.checkForExpiredSession(linkURL, showUserFile, OnFail);
}

<!--note, this is the PageMethods.checkForExpiredSession prototype, cannot seem to find the code -->
PageMethods.checkForExpiredSession= function(linkURL,onSuccess,onFailed,userContext)





And by the way, the login page is the same as this page, and the code that is delevered to the browser does not change if I am logged in or not.

Thanks (yet again)
-wired_al
Author

RE: Solution Found


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 15:06
Oddly enough, it i was this line of code that helped me find my answer last night at around 1:00 or so...
Code

<script language='javascript'>alert( "Could not find file 'C:\inetpub\TempUser\xxxxxxxxxxFFVFiles.zip'." );</script>




I figured I couldn't access the TempUser directory since the default directory for webhosting is c:\inetpub\wwwroot, but thought at this point, anything was worth a shot. So after clicking the link for the date, I went to https://website.gov/TempUser/xxxxxxxxxxFFVFiles.zip and it began to download. I figured that I would let the WebBrowser control authenticate automatically, click the links, and then I would use the WebClient class to download like this.

Code

//after authenticating and clicking link for todays date...
using (WebClient wc = new WebClient())
{
    wc.DownloadFile     ("https://website.gov/TempUser/xxxxxxxxxxFFVFiles.zip", "c:\\temp_data\\"+currentdate+".zip");
}





It works, the reason I have to click the link for the date I want first is because the .zip files are non-existant until I click the link and the javascript function runs, then the .zip file is created and I can download it.

Thanks for your help
-wired_al
Author

RE: Downloading from a https webserver

stealth-
Member



Posts: 1003
Location: Eh?
Joined: 10.04.09
Rank:
Mad User
Posted on 11-10-10 19:23
Glad you got it working.

Out of curiosity, what exactly is all this for...?


The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
http://www.stealt. . .
http://www.stealth-x.com
Author

RE: Downloading from a https webserver


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 11-10-10 20:47
stealth- wrote:
Glad you got it working.

Out of curiosity, what exactly is all this for...?


The people that have hired me buy potatoes from the farms, then they sell them to people who sell potatoes. Up until now, they have been told how much money they should be paid by the people that they sell potatoes to because they are too lazy to look through the xml files that have all the weights and types of potatoes and such. I heard about this, talked to them and my program downloads the data, parses it, dumps it into a sql database, then generates reports and delivers it to the president and owners of the company. They can then tell how much money they should be getting paid.

If you are from my state, you probably know it. What other state would have a state run website dedicated to the movement of potatoes?
Author

RE: Downloading from a https webserver

fashizzlepop
Member



Posts: 482
Location: Old folks home.
Joined: 08.04.08
Rank:
Moderate
Posted on 15-10-10 08:49
Idaho, you 'da pimp.


"The definition of insanity is doing the same thing over and over again and expecting different results.
~Albert Einstein~


csullivan.codeinspire.net/images/boomsig2.png
fashizzlepop@gmail.com http://csullivan.codeinspire.net/