Advice for hosting a quite big website

Can you give me any advice what hosting should I consider?
Distributor has ~ 1mil products and for this I would also need quite complex sql scripts to for linking products to categories and mapping filters. That's a lot of data. I'm thinking about VPS, but what specifications server would I need?

I think litecart engine is quite forgiving in terms of hosting performance. I had impressive results with shared hosting for small websites with only 100-200 products.

For this particular website I need something reliable, but I haven't considered the scale of the project and my current hosting isn't even capable importing data itself. Thanks for advice.

Thanks for advice.
tim
Without knowing the details of the project myself I can only recommend hiring an experienced developer to assist you in the best solution possible.

Perhaps the best performance could be achieved by a dedicated server running Nginx, MariaDB and PHP 7.1. Importing data segmentally and setup proper indexes for the data. Even indexes for fulltext searching might be necessary.

To update the catalog in the future I recommend fetching only the updated products over an API and a background job. This can be done by having the API providing a list of products filtered by updated_since date.

If you have a high amount of categories you might have to rebuild the way you select categories when editing products. Displaying them all in a selectable tree structure will cost performance and is hard to scroll through.
Do you happen to know what might be causing this error running Nginx, MariaDB and PHP 7.1?

I got this right after install. Developer can't seem to find a solution.

Fatal error
: Uncaught Error: Call to undefined function json_encode() in /usr/share/nginx/html/includes/library/lib_reference.inc.php:46 Stack trace: #0 /usr/share/nginx/html/includes/library/lib_customer.inc.php(173): reference::__callStatic('country', Array) #1 /usr/share/nginx/html/includes/library/lib_customer.inc.php(42): customer::identify() #2 [internal function]: customer::initiate() #3 /usr/share/nginx/html/includes/classes/system.inc.php(38): forward_static_call(Array) #4 /usr/share/nginx/html/includes/app_header.inc.php(35): system::run('initiate') #5 /usr/share/nginx/html/index.php(16): require_once('/usr/share/ngin...') #6 {main} thrown in /usr/share/nginx/html/includes/library/lib_reference.inc.php on line 46

tim
Did you try

CentOS
sudo yum install php71-json

Debian
apt-get update
apt-get install php7.1-json

The package name differs on which repository you are using.

Make sure you have installed all the common php extensions such as mbstring, xml, mysqlnd, gd2, zip, and more..
Thanks, I've managed to run Litecart on this new server but that doesn't seem to help at all.

I think the only solution here might be dedicated server, but that is way out of my budget for now.

Do you know any websites that are using Litecart for such high amount of products?
If you have a high amount of categories you might have to rebuild the way you select categories when editing products. Displaying them all in a selectable tree structure will cost performance and is hard to scroll through.

Could you be more specific? It seems that I have this problem already.
tim
Yes, fetching a categories from the database and creating a hierarchy is a lengthy process for PHP or MySQL.

When you edit products you can mount a product to a category. This is convenient. What is not convenient is if you have 500 categories. Then it takes time to generate the list, and it's not convenient to navigate in it. This selectbox is output by form_draw_categories_list in func_form.inc.php and would prefereably be replaced by some ajax search function.

Then there is the top menu which lists which categories to display in the top. If you are not displaying categories here be sure to remove the code that queries the database for categories in includes/boxes/box_site_menu.inc.php.

Fot the left column category tree you might e able to speed up the query for categories by removing the condition for the column 'dock'.

When it comes to large sql tables, you need to add some custom indexes and partitions that suits your data the best.
https://stackoverflow.com/questions/1579930/what-is-mysql-partitioning
In answer to the original question, it depends on the number of visitors you are expecting the website to see.

For all intent and purpose a VPS is a dedicated server its actually just a container with its own dedicated resources, some are a kind of hybrid that will allow you to suck up some of the spare resources if you have a spike but only for a short period of time.

A dedicated server is more costly but at the same time, the entire physical server is yours. The only limitation is set by the hosting company, normally traffic limits that kind of thing.

If you find that a large dedicated server cannot run your website efficiently then you are looking at a more complex server configuration. By this point the cost of such an undertaking in comparison to what you are making form the store should be negligible and easily worth the expense.

You have options.
Move to the cloud (pay for what you use but expand very easily with no down time),
Rent more servers and load balance them (set costs, expanding means renting servers),
Buy servers load balance them: This comes with options of its own, collocate, install at own premises etc etc

For me as i run a server on my home system the option is buy and load balance. Yes its a bigger initial outlay but after that its just using resources im already paying for, internet, electricity. Future costs come from expanding and/or repair of the server(s).

My next move is load balancing. for a small site you could easily setup on 5 servers: (locate your mail/dns away from your own network some very cheap vps can handle the job perfectly).

1 load balance server (2 would be better but one is a good start)
2 web servers
2 database servers

The load balanced setup works by directing traffic to the least loaded server, whether that is web traffic or database traffic doesnt matter, the load balancer can monitor all of the servers (with 2 they also monitor each other). and send traffic as appropriate.

The benefit of two load balancers is that your website will have massively reduced down time, and better still you can do this across multiple locations. That means you could set up a in 2 or 3 datacentres and your website will never ever go down (save for some catastrophic world shattering event in which case you wont care ;) )
@tim, can you please comment about this, I'm not sure I follow:
Importing data segmentally and setup proper indexes for the data. Even indexes for fulltext searching might be necessary.


I've already made a huge improvement. Website most likely will be ok, as it was motly configuration o MySQL and PHP, that was slowing it down. I'm still have a search problem which performance hasn't been affected and it's very slow. Also where should I look if I want search to work with sku's, gtin, maybe search text in the middle of the phrase and even to search in attributes. Or would that make things even worse?

@user4147 thanks for the input. I've tried with two servers and it seems that my server has enough resources for now. It was configuration that was slowing website down, but hosting company said it might be useful in the future, but that really depends on the traffic, so I'll guest we just have to wait and see. :)
tim
@Dodo Sure, by segmental imports I mean instead of importing a large complete set of data you split that data into several files/chunks (sort of the same idea as pagination). Say you extract 5000 products rows per file.

With fulltext indexes I mean replacing something like the following:

WHERE (name like '%something%' OR description like '%something%')

...with the following instead...

WHERE MATCH (name, description) AGAINST ('namedescriptionindexkey' IN NATURAL LANGUAGE MODE)

Set up by

ALTER TABLE tbl_name ADD FULLTEXT INDEX `namedescriptionindexkey` (`name` ASC, `description` ASC);

https://dev.mysql.com/doc/refman/5.7/en/fulltext-search.html

For regular indexes:

ALTER TABLE table_name ADD INDEX `your_index_name`, (`some_columns_name_used_in_your_where_clauses`, ...);
Ive not had the chance to look into it as yet so il just ask, what about caching pages that are frequently loaded but rarely change.
tim
Sure caching can also be adjusted to suit your needs. In LiteCart you can specify the maximum age for cached content in cache::get() and cache::capture(). If cache is within max age then a result will be returned.
Does LiteCart searches products which are disabled? (status=0) I mean is it any difference between disabled and deleted products. Also would I benefit from Ajax search like in admin panel?
tim
@Dodo The frontend queries the database for products having a value for status (meaning is not 0).

Status 0 (Disabled) means the product remains in the database but should be treated like if isn't. If it's removed then the database row is removed as well.

Accessing a disabled product triggers HTTP 404 Not Found on the product page. Accessing a removed product triggers HTTP 410 Gone.

In what aspect do you wonder if you would benefit from ajax search? Performance? Ajax searching triggered by keyup would not reduce your server load or performance, rather the opposite.