mysql query optimization (wrong indexes?) avoid filesort

Question

I will try to explain myself quickly. I have a database called 'artikli' which has about 1M records. On this table i run a lots of different queryies but 1 particular is causing problems (long execution time) when ORDER by is present.

This is my table structure:

CREATE TABLE IF NOT EXISTS artikli (
  id int(11) NOT NULL,
  name varchar(250) NOT NULL,
  datum datetime NOT NULL,
  kategorije_id int(11) default NULL,
  id_valute int(11) default NULL,
  podogovoru int(1) default '0',
  cijena decimal(10,2) default NULL,
  valuta int(1) NOT NULL default '0',
  cijena_rezerva decimal(10,0) NOT NULL,
  cijena_kupi decimal(10,0) default NULL,
  cijena_akcija decimal(10,2) NOT NULL,
  period int(3) NOT NULL default '30',
  dostupnost enum('svugdje','samobih','samomojgrad','samomojkanton') default 'svugdje',
  zemlja varchar(10) NOT NULL,
  slike varchar(500) NOT NULL,
  od_s varchar(34) default NULL,
  od_id int(10) unsigned default NULL,
  vrsta int(1) default '0',
  trajanje datetime default NULL,
  izbrisan int(1) default '0',
  zakljucan int(1) default '0',
  prijava int(3) default '0',
  izdvojen decimal(1,0) NOT NULL default '0',
  izdvojen_kad datetime NOT NULL,
  izdvojen_datum datetime NOT NULL,
  sajt int(1) default '0',
  PRIMARY KEY  (id),
  KEY brend (brend),
  KEY kanton (kanton),
  KEY datum (datum),
  KEY cijena (cijena),
  KEY kategorije_id (kategorije_id,podogovoru,sajt,izdvojen,izdvojen_kad,datum)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

And this is the query:

SELECT artikli.datum as brojx, 
     artikli.izdvojen as i, 
     artikli.izdvojen_kad as ii, 
     artikli.cijena as cijena, artikli.name 
FROM artikli 
WHERE artikli.izbrisan=0 and artikli.prodano!=3 
     and artikli.zavrseno=0 and artikli.od_id!=0 
     and (artikli.sajt=0 or (artikli.sajt=1 and artikli.dostupnost='svugdje')) 
     and kategorije_id IN (18) 
ORDER by i DESC, ii DESC, brojx DESC 
LIMIT 0,20

What i want to do is to avoid Filesort which is very slow.

Answer 1

The problem is that you don't have an index on the izdvojen, izdvojen_kad and datum columns that are used by the ORDER BY.

Note that the large index you have starting with kategorije_id can't be used for sorting (although it will help somewhat with the where clause) because the columns you are sorting by are at the end of the index.

Answer 2

It would have been a big help if you'd provided the explain plan for the query.

Why do you think its the filesort which is causing the problem? Looking at the query you seem to be applying a lot filtering - which should reduce the output set significantly - but none of can use the available indexes.

artikli.izbrisan=0 and artikli.prodano!=3 
 and artikli.zavrseno=0 and artikli.od_id!=0 
 and (artikli.sajt=0 or (artikli.sajt=1 and artikli.dostupnost='svugdje')) 
 and kategorije_id IN (18) 

Although I don't know what the pattern of your data is, I suspect that you might get a lot more benefit by adding an index on :

kategorije_id,izbrisan,sajt

Are all those other indexes really being used already?

Although you'd get a LOT more bang for your buck by denormalizing all those booleans (assuming that the table is normalised to start with and there are not hidden functional dependencies in there).

C.

Answer 3

Actually, the order by is not the basis for the index you want... but the CRITERIA you want to mostly match the query... Filter the smaller set of data out, you'll get smaller set of the table... I would change the WHERE clause a bit, but you'll know your data best. Put your smallest expected condition first and ensure an index is based on that... something like

WHERE  
       artikli.izbrisan = 0 
   and artikli.zavrseno = 0 
   and artikli.kategorije_id IN (18)  
   and artikli.prodano != 3
   and artikli.od_id != 0
   and (    artikli.sajt = 0 
        or (     artikli.sajt = 1
             and artikli.dostupnost='svugdje')
           )

and having a compound index on (izbrisan, zavrseno, kategorije_id)... I've mode the other != comparisons after as they are not specific key values, instead, they are ALL EXCEPT the value in question.