Can the file limit be increased?

[digx]

I understand why there is a space limit but why a file limit? I want to install 2 copies of PHP-Nuke on my server and os-commerce. It will not exceed my space limit but will reach my file limit.

Could someone please explain why there is a file limit? It is really keeping me from adding some things to my website. Thanks in advance and Happy Holidays!

[Naveen]

The file limit is due to the operating system powweb uses ( Free BSD ).

Search the forums and you'll have more information on this.

[Sgeine]

Its not due to the OS we use. Your files aren't even on a FreeBSD machine. They're on a NetApp. In fact, operating systems don't have anything to do with how many file descriptors you have. It is handeled on a much lower level. When you format a drive you typically do it with 4k blocks. You get one file descriptor for each 4k block. Meaning on a 20g drive you can have a total of 5 million files. So if you create 5 million 1 byte text documents (5megs total) your drive will be full.

On a NetApp it is a little different. The number of file descriptors can be set on the fly. However, when you increase the number of file descriptors the file allocation table grows proportinately. The larger the FAT grows the worse the performance of the filer gets because its now searching through a FAT 10gigs (for example) in size for the block location of your file. Not very efficient. This is why that limit is in place.

[Rally]

Originally posted by Sgeine
Your files aren't even on a FreeBSD machine. They're on a NetApp.

Wow. Interesting. How does that work? Is it explained somewhere? So is andromeda.powweb.com an application server running FreeBSD with Apache as the http server and calling the files from the NetApp file server?

I love nitty-gritty details. I learn so much reading these forums.

[tbonekkt]

You've got the basic understanding Rally. Check out this page on PowWeb's homepage: http://www.powweb.com/company/technology.html

[digx]

What's the real advantage of this? Why can't there be a regular machine holding the files?

[Skunkboy]

Its all fine and dandy. I somewhat understand why there's the limit; but, honestly, how many people are going to put a million 1k files on their site? Now days, with highspeed net, great cameras, great image software, and great multimedia in general, it is rare to find anything less than 10k and 100k or better is not an uncommon file size.

My question then is:
If we keep getting increases in space, why can't we get an increase in file count? We get the space because the general file sizes are increasing? But what about those that aren't using the big multimedia files but still have quality content.

I don't use FrontPage Extentions; if I did, I would've blown the limit to shrapnel a long time ago. I appreciate the space and all but will have problems before long with file count. I'm not saying to tripple the limit or anything but some increase would be nice.

I understand the concern for lag but at the same time, PW says they're operating in the green... if that's the case, can't the extra servers be provided to make up for this change? And honestly, would an increase to even 25,000? My site stats as of now: 433.17 MB / 12177 Files. Won't be long and I'll be fried. I don't want to leave PW but it won't be long till I won't be able to update unless the limit is increased. 20,000?

Happy Holidays
(no, I'm not angry... just stating facts)

[coan.net]

Advantages: Having the services spread out on different machines for one helps spread out the load. Another reason is if lets say the FTP server goes down, now all it affects is people using the FTP, and does not also take down the file server & the web servers.

Plus it would seem if the FTP server or webserver goes down, all they would have to do is replace it with another machine without worring about having to restore data - since it would be save on the filer server. (Which I'm sure is well protected from being taken down easily.)

[tbonekkt]

It's my personal opinion if your site contains more than 15,000 files (and you're not using Front Page), a shared hosting environment is probably not the best hosting solution available.




Also wanted to add that if a dedicated hosting solution is out of the question, you can always purchase a second package and link to files between the two.

[danpadams]

What is the official limit on the number of files? I know with some of the stuff I am developing for people to use it does not matter, but I am interested in the actual amount anyway.

[alphadesk]

Dan

From the policy page ..

"Each package has a 15,000 file limit."

but I believe it's actually 15,346

Either way if you hit 15,000 you really need to look at the structure of your site.

[Skunkboy]

To view your file limit:

Ops / Packages / FTP

Tells your file count and used space

[Sgeine]

Skunk,

"I somewhat understand why there's the limit; but, honestly, how many people are going to put a million 1k files on their site"

you'd be surprised some of the things we've encountered people doing. We asked ourselves something similar when we put the limit in place, "how many people could use more then 15,000 files?however i think you may have missed the purpose of my example. you also said:

"PW says they're operating in the green... if that's the case, can't the extra servers be provided to make up for this change?"

This makes it pretty clear it was missed. the filer holds the files, like i said earlier there are no customer files (outside of FP extensions) held on the servers. adding more servers does not change things. whether there's 100 servers or 50 servers it doesn't make a difference, the netapp is the one with the large FAT and the thing that has to go through and search through that FAT. think of the netapp as a windows share or a mounted hard drive and you'd be closer to what's going on.

also, what coan said was exactly right. at the very least it allows us to distribute our services and make our customer files more secure. from the start it gave our customers fewer ways to cause outages on any individual server. It also allows us the possibility of load balancing the webservers which will increase uptime exponentially.

[mdastous]

Unfortunately, I have to agree with Skunkboy. I reached my 15000+ limit last night. I was trying to upgrade my distance learning site (keeping the original folder as a backup on the server) and I reached the limit an hour into the upgrade. I had to delete the entire upgrade and revert to the backup.

I do not have the luxury of moving to another service. As a K-12 educator, I use Powweb for my classroom materials because it is affordable. I cannot afford to start my own server, or look into independent hosts. I have gone that route before and it cost me thousands of dollars a year.

I do use Frontpage extensions, and the additional files that are placed in the _vti folders most certainly each up much of my allotted space, but how can I change that? I am under the impression that if FP extensions are on, these directories must exist. Is this true? If not, how do I disable them?

I have another site hosted with powweb that is using only 1100 files of the 15,000. Is there anyway that I can "share" this file space? They are on the same server.

Any input to help solve this problem would be greatly appreciated. The second semester starts in a week and I would like to fix this before class starts on the 6th.

Happy Holidays all!!!

Marc Dastous
http://www.gis4u.com/
http://www.socialstudies4u.com

[FlyinV]

I don't know if you will like this idea, but you could move away from Frontpage and the frontpage extensions and have a lot more file space.

You can still use Frontpage to make your pages, just do not use any Frontpage extensions or the upload feature. Upload your pages thru another FTP client.

[Rick_E]

If you have done a lot of editing of your site, there may be quite a few of the FrontPage files that are obsolete but still left on the server. You may want to wipe the server clean then republish. You might end up with a lower file count. I've found the file count on the server usually runs about 2.5 times the file count listed in the FrontPage report. If yours is above 3.0 to one you may benefit by cleaning the server and republishing.

>>I do not have the luxury of moving to another service.>>
There are other low-cost hosts who do not have a file limit.

[mdastous]

Ok;

Is it possible then to delete the contents of the _vti folders without destroying the FP utility?

FYI, I originally created most of the pages using an HTML editor, but moved to FP for the ease of use; one stop shopping if you will ;)

Thank you for the input, I may just have to use WSFTP and WinXP's Network Places feature to facilitate the additional space.

Marc Dastous
http://www.gis4u.com
http://www.socialstudies4u.com

[Croc Hunter]

Like Rick_E said.. delete the lot.. _vti if you want. But so long as you are sure you have a local copy of all the files you want to keep lined up and ready to re-load.. delete the lot.

[mdastous]

So you are saying delete EVERYTHING and re-upload all 400 MB???

There has to be a better way? That will take hours, even on my broadband connection.

Please advise.

Marc

[Rick_E]

Get it started at night and in the morning it may be finished. You don't have to watch it :)

[Croc Hunter]

Yes everything.. look in FP help. Only some components need FP Extensions enabled to work properly. If you don't use them turn off FP Ext in OPS and your version. Wipe your site. All of it. And reload at a conservative say 27kbps total eta.. 3.5hrs.

FP Ext creates heaps of files. If disabled these won't work.. through FP.

Hit Counter (Insert menu, Component submenu)
Confirmation Field (Insert menu, Component submenu)
Include Page (Insert menu, Component submenu)
Scheduled Include Page (Insert menu, Component submenu)
Categories (Insert menu, Component submenu)
Search Form (Insert menu, Component submenu)
Additional Components (Insert menu, Component submenu)

[lsimpson4]

I run a lot of pictures in my setup so I have one thumbnail to every pic. So far I'm 193.30 MB / 4244 Files but down the track this will void approx. 40% of my space. I understand the technical limitations but Powweb you promote the fact we get 1024Mb (1Gb) which is why I've hung around. But now I learn I have approx. 580Mb. Can't you change your File Server to be spread across move hardware to avoid the load on the one machine (virtual environment)? Technical is one thing but we are the customers and we want our 1Gb please.

Thanks in advance and happy holidays

Regs
Lachlan

[Skunkboy]

I've posted here already but I'll add my name once again.

A gig is offered here but yet we won't be able to use it. The only way to reach a gig and not go over the limit would be to press the boundries and risk jail time by posting videos or mp3's or something that in some way would probably cause infringement problems somewhere.

My vote again for an increase. Maybe not to a million but 20,000, 25, 000? Something?

The majority of sites out there, weather like ours (image based) or not have the same setup for their images as well. It is best for load time to use smaller images on the pages themselves and give a user an option to view whatever the graphic is available in a larger format. In other words, it isn't like we're trying to break the limit.

Again, it is understood why there is "a" limit in place; but, it would be nice to see something of an increase. Our space is increased but why? as a teaser? It is near impossible to get there.

::edit::
woohoo, they now even gave us the bandwidth to transfer gig sized sites... so we've got the space, we've got he bandwidth, now we still need the file count.

- I still wouldn't leave, there's too many good things about here to bother doing that but it would be nice to be able to use everything that we're given.

[el2ik]

Yeah, I too am also worried about the file limit which I didn't even know about until recently...

For the new site that I'm co-running, we're expecting a LOT of people... so with all the forums (no, not just 1 forum, therell be a lot of seperate ones), all the php stuff (running on nuke 7 modified a lot, except a lot of nukes, its not just 1 site, lots of "mini-nuke" sites on it for each city), and the image uploading where people can review each other pictures (of cars, its a racing site), reviews, etc etc it may be a problem for me

I don't know I hope 15,000 is enough, but I can EASILY see me having to buy at least 2 more packages just because of the file limit, oh well at least it's a good price

Plus I'd rather just buy a few more packages here then go anywhere else, Powweb offers so much more for so much less, and its reliable, seriously have everything here, but if you guys somehow found a way to increase file limits it'd be appreciated, but it's not that big of a deal

[tbonekkt]

Originally posted by el2ik
Yeah, I too am also worried about the file limit which I didn't even know about until recently... For the record...it's in PowWeb's policy (http://www.powweb.com/policy.html) that we all stated we read, understood, and accepted when we signed up for hosting.

[lsimpson4]

That may be true but Powweb use the 1Gb as a selling tool on their page. Should make the trade off a bit clearer too :)

[FlyinV]

People should read the fine print....

You get the space so its not like Powweb is lying about it.
Using it fully is a completely different issue.

[lsimpson4]

Disappointing but true.

Happy New Year everyone.

[YvetteKuhns]

I think the 1 gig of space can be consumed faster if most of us used larger files, but since we are not, the file limit approaches sooner. It may seem misleading, but it really isn't. It is unfortunate for those who reach the file limit first.

For such a low price, purchasing a second web hosting package is worth it. The bad thing is you need a second domain name to point to it. It doesn't cost much, but it is an extra concern. Most of us won't have to worry, but for those that do, sorry for the extra trouble. Still a good value here.

[kanwulf]

For the record...it's in PowWeb's policy that we all stated we read, understood, and accepted when we signed up for hosting.

It wasn't in there when I purchased my package, but then again neither were these forums or OPS and the file limit was around 100Mb. Which I am still under {80Mb} but I have over 13,000 files.

I have converted my thumbnails to base64 and placed them all in a single file and call them using PHP. I have made liberal use of symbolic links. These two tricks have helped me reduced my file count by a couple of thousand.

[tbonekkt]

Originally posted by kanwulf
It wasn't in there when I purchased my package, but then again neither were these forums or OPS and the file limit was around 100Mb. Which I am still under {80Mb} but I have over 13,000 files. Very good point, kanwulf. Let me rephrase...for any semi-recent new customer, it was in the policy. :)

[lsimpson4]

Originally posted by kanwulf
I have converted my thumbnails to base64 and placed them all in a single file and call them using PHP. I have made liberal use of symbolic links. These two tricks have helped me reduced my file count by a couple of thousand.

kanwulf could you point me in the right direction for some info on your setup? Sounds interesting...

[kanwulf]

I remember when they changed things and added the limit. I tried to upload some files one day and was locked out of my FTP. I couldn't upload anything until I deleted enough files to be below the limit.


Here is a link the the source file of some of my thumbnails. Unzip it and view it in you favorite text editor. There are 60 or so thumbnails in this one file.
sample (http://jeeptalk.org/thb.zip)


I have a converter I use on my machine. You can also upload it and use your powweb accout to convert images to base64. I'll have to go dig it out and stop back and share it with you.

[lsimpson4]

Excellent!!! Any info you could offer in setting it up and calling it would be appreciated. I just took a look at your site love the pics!!! Thumbnails look great I think I'll need to start converting to this system to reduce my file count.

[kanwulf]

You can get a script to convert to base64 here.
http://php.holtsmark.no/base64img/

It spells things out a little better than my chop work script that I would have to explain.

If you need any guidance don't hesitate to ask. I don't stop in here to frequently but I'll get an email if you private message me.

[lsimpson4]

Thanks Champ I'll look into it.....

[user00265]

Originally posted by kanwulf
You can get a script to convert to base64 here.
http://php.holtsmark.no/base64img/

It spells things out a little better than my chop work script that I would have to explain.

If you need any guidance don't hesitate to ask. I don't stop in here to frequently but I'll get an email if you private message me.
Also it would be another idea to store them on a mysql table. I have seen this done before, so it is possible. It will count against your space, but not at your file count, and with the above method, if the file get's TOO big, then the script will be ALOT slower than a simple mysql app. well written.

Again, this is just a suggestion. That base64 method with mysql plus a calling script WELL written is worth more than a BIG file. Even tho, this is NOT recommended for big images (bigger than 200k in my opinion).


HTH,
Samuel