Recent

Author Topic: Won't load Apt.Dat.  (Read 13127 times)

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #60 on: January 12, 2019, 05:47:44 am »
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

wp

  • Hero Member
  • *****
  • Posts: 5444
Re: Won't load Apt.Dat.
« Reply #61 on: January 12, 2019, 07:06:27 am »
My demo program of reply #30 loads and displays all records of apt.dat within 10 seconds. Just change the const FILENAME near line 50 to the path to apt.dat.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

GetMem

  • Hero Member
  • *****
  • Posts: 3414
Re: Won't load Apt.Dat.
« Reply #62 on: January 12, 2019, 09:14:17 am »
The file is loaded in 2 seconds at my side.  :) On a slower computer some feedback(progress) to the user would be nice.  @JLWest please study @wp's code. It will load the data into a dynamic array, then display in a ListView. All you have to do is add the filter.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #63 on: January 12, 2019, 10:02:57 pm »
My demo program of reply #30 loads and displays all records of apt.dat within 10 seconds. Just change the const FILENAME near line 50 to the path to apt.dat.

Oh that would be on the small data set. Yea.

will try #30 and #34.

As I said I'm actually loadind the big nail but not seting the array in one set.

Say what! Is the last record '99'? I got it loaded I think but not in 10 seconds.
I'll go back and incorporate #30. I think it must have taken 20 to 30 min on my machine. Maybe I need a tuneup.

« Last Edit: January 12, 2019, 10:34:19 pm by JLWest »
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

wp

  • Hero Member
  • *****
  • Posts: 5444
Re: Won't load Apt.Dat.
« Reply #64 on: January 12, 2019, 10:40:36 pm »
Is the last record '99'?
The (0-based) index of the last record is 7,970,311 and the first number (I call it "ID") is 99.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #65 on: January 12, 2019, 10:42:43 pm »
You loaded it.
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #66 on: January 13, 2019, 10:10:16 pm »
Modified code to run Example #34. Loads the 62 records set fine and shows the 99 record in a edit box.

Pointed it at the the 7.9 and. It gives the correct record count (BlockSize).

A few seconds after pressing the load button it lights the top right X (exit) red reports a not responding in the title bar a second or two later and I think out to lunch.

Guess what, I'll be. It was running while I was typing this. Went over to close thinking to make a final change. and I have the big nail loaded. 99 in the TEdit.text and the correct number of records reported in the label2.caption which match the Blocksize.

I don't know how long it took, a bit, 5 to 8 min maybe. Haven't been able to  make the timer work and need to research.

Now how to manage all that data edit and save.
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

josh

  • Hero Member
  • *****
  • Posts: 719
Re: Won't load Apt.Dat.
« Reply #67 on: January 14, 2019, 02:09:54 am »
Well I bit the bullet and downloaded it,
Had an hr or two spare and came up with this, no special load routine just standard file i/o

If you click the R:\AIRDATA\apt.dat label at the top; you can point it to the location of the file.

It read in analyzes and the data.
You can select via country then city, or from a complete list.

I have done some filtering and checking for duplicates which effects the speed a little.

Once you have selected the airport, the airport details shoudl popup; as all this is in memory.
In the name of the airport details box at the end is the actual filelinenumber that any airport details would reside; so
you would then read the main apt.,dat file until you got there and then scan forward to the details you after.
I have not added this in, but should be easy enough.


I have not fully tested, just a test to see how complicated this actually is..

Also not the neatest code;

WIth the data loaded and everything poulate, should take a total of about 25MB.
« Last Edit: January 14, 2019, 02:18:58 am by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #68 on: January 14, 2019, 03:47:21 am »
@ Josh

Don't fully understand what your saying but I'll run it up and try and figure it out.


Thanks.
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #69 on: January 14, 2019, 05:52:27 am »
@Josh

Had a response typed in and got logged out and lost the typing. Hate that about this system. So here goes again.

Do you by any chance like a little color on you screens?

I can't tell whch file you are pointed at, but I tested it on my test file, worked fine and then the 9.7mil file. Half way thru it took a big hit on performance and I'm not sure if it loaded all the records.

The last record in a Apt.dat file is just '99'.

I'll add a counter and display the last record to a label1.caption to check.

In the lower maroon box there is a number at the top right 14,666. I think that may be the entries in the box which is the airports. Should be 35,000+.

You display City country which is a problem. Most large cities have several airports. Phoenix KPHX, Dear Valley  Busiest GA airport in the world, Gelipse field, Mesa, Chandler, Scottsdale. In order to sort the mess avaiation use ICAO Codes. Guarantee unique code by country. US code starts with a K or a P in Alaska. KPHX Phoenix, KLAS Los Vegas, PANC Anchorage. Canada with a 'C'  CYUL Monteral.

Where I'm loading all 9.7mil records in a dynamic array they are stored in an array of records.
Fline is the text record or text line. I added RCDID which is = to Copy2Space(Fline); and ICAO which is the 5th field in of the RCDID '1', airport, '16' seaplane base and '17' heliport. It is the ICAO code "KPHX'. Thinking it will make searching thre array of records easier. but it does create a redundancy of data. The other two important ones '1300' and '1301' are the ones I need to edit. All the rest are just baggage I have to carry.

 type
 PData = ^TData;
  TData = record
  FLine  : String;
  FIndex : Integer;
  RCDId  : String;
  //other data if needed
      end;
   
Unique way of setting the filename path. Took me a bit to figure out. Not a bad idea. Do you really have an R drive?

Thanks
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

josh

  • Hero Member
  • *****
  • Posts: 719
Re: Won't load Apt.Dat.
« Reply #70 on: January 14, 2019, 10:26:20 am »
DO you only access by ACAO code?

as line 407400 in aptdat

I do not see ACAO code
1   1212 1 0 XLRBANE BANESTI PRAHOVA
1302 city BANESTI
1302 country ROMANIA
1302 datum_lat 45.068673
1302 datum_lon 25.806558
1302 state PRAHOVA
1302 transition_alt 3000
1302 flatten 1
« Last Edit: January 14, 2019, 10:30:48 am by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

josh

  • Hero Member
  • *****
  • Posts: 719
Re: Won't load Apt.Dat.
« Reply #71 on: January 14, 2019, 01:42:36 pm »
Hi
Altered during lunch.

Modified code for just acao, visible.
Storing the 1300 and 1301 in memory, when you select a ACAO code, if it contains 1300/1301 then they appear in the grid, you can then edit in grid, the center memo keeps a track of what you have altered ( visually at the moment); but would reaaly be stored internally.

As ACAO is in Listbox, you can when selectiiing them just press the keys for KLAX; to save scrolling.

Yes I have an R drive it is used for 'Rubbish' stuff, that is never backed up and does not matter if it gets deleted.
My Projects are on P Drive
Lazarus Installs are on L Drive.

All seperate physical drives.
« Last Edit: January 14, 2019, 02:50:47 pm by josh »
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

josh

  • Hero Member
  • *****
  • Posts: 719
Re: Won't load Apt.Dat.
« Reply #72 on: January 14, 2019, 02:16:21 pm »
Just moded the upload.
Development Installation Lazarus 1.3, FPC 2.7.1,Windows 7/8 32/64, OSX, *nix

Test Environment Lazarus & FPC Trunk on Windows and OSX (Cocoa Mainly on OSX). Testing also Crosscompile windows to OSX.. 
Any posts made from 2015 will be based on Lazarus Trunk.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #73 on: January 14, 2019, 05:55:59 pm »
Yes we only access by ICAO. Although in X-Plane when you select an airport to start from  you can type out a city and sometimes get the right airport. But it's a poor search engine and usually comes back with nothing.

Here is the first record in the live real data file:



1    460 1 0 LELL Sabadell

The '1' is the record ID. Indicate a land based airport. (About 30 different  Record Id).

'460' I think is the average elevation of the airport. Other records give the elevation of the touchdown zone elevation of each runway.
 
'1' and '0' are what they call depreciated fields. Fancy term meaning not used for anything in X-Plane. Maybe in other applications. This file has been around for a long time and it is is my understanding it is used and adapted for use in many commercial applications. 

LELL is the IACO code for the airport at Sabadell.
It is always the fifth field in the record in the '1', '16' and '17' record.

Sabadell Name of City where Airport is located.

On the record you posted:
 
1   1212 1 0 XLRBANE BANESTI PRAHOVA

The ICAO committee has allowed legacy codes exceeding 4 characters which is the standard now. XLRBANE is the ICAO code. It is the 5th field in.

« Last Edit: January 14, 2019, 06:01:46 pm by JLWest »
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.

JLWest

  • Sr. Member
  • ****
  • Posts: 390
Re: Won't load Apt.Dat.
« Reply #74 on: January 14, 2019, 07:15:00 pm »
It's actually very neat. I think I would add a TEdit and enter the ICAO code for the airport I want.

The lower two boxes are good. I would set focus on the 1300 record and the corresponding 1301 record should get focus also. Easy enough as you indexed them.
 
A double click on any 1300 would move both records (1300 and matching 1301) to a restructured blue box for editing. This part is a little tricky on how you would present it, put it back together.

Notice how a lot of the 1300 say 'HeavyjetsProps' that all computer generated. If you really go to KLAX and look there are a lot of gates that can't take a heavy Jet, A380, B747. That's what wrong with the data.And the 1301 are even less useful. C airline. Who? AAL DAL SWA, CAP BAW.
JLWEST 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
500 GB SSD
3 Terabyte conventional disk space.