View Issue Details

IDProjectCategoryView StatusLast Update
0000039opensim-SCBugpublic2019-07-17 19:49
Reporteronefang Assigned Toonefang  
PriorityurgentSeverityblockReproducibilityalways
Status resolvedResolutionfixed 
Product Version0.9.0.1 RC0 
Target Version0.9.0.1 RC0Fixed in Version0.9.0.1 RC0 
Summary0000039: Half the assets are missing.
DescriptionI think this is a ROBUST issue. I suspect it's not reading the file assets properly, coz it seems to be new assets not turning up.
TagsNo tags attached.

Activities

onefang

onefang

2019-07-17 12:40

administrator   ~0000077

Last edited: 2019-07-17 12:44

I think I have it figured out. The FSassets system is slicing the file names differently when it stores them.

Several years ago, using OpenSim 0.8, I switched to using FSassets. When trying to find an asset, it first looks in FSassets, if it can't find it, it looks in the old assets database table, then copies the asset to FSassets. If it's creating a new asset, it creates it in the FSassets directory structure, but doesn't touch the database.

OpenSim 0.9 is working the same way, the difference is in the directory structure it is using for FSassets. Since it is looking in a different place, it can't find any assets created since I switched to FSassets, only the older ones in the database. And as usual, if it finds an old one, it copies it to the FSassets directory structure.

Here's an example. On November 10, 2016, an asset with the ID of 000C85B11C1FA989292C5564E9DD0004381041A9C7348B80B7382725B09DD216 was accessed. It wasn't in the 0.8 FSasset system, but it was in the database. So OpenSim 0.8 created this directory structure -

AssetFiles/data/000/C85/

And added this file into that directory -

000C85B11C1FA989292C5564E9DD0004381041A9C7348B80B7382725B09DD216.gz

Note that the directory structure is based on the first 6 digits of the ID number, which is hexadecimal. An OpenSim 0.8 FSassets directory will be filled with these three digit named directories, each of those having some number of three digit named directories, and those having the actual asset files.

In OpenSim 0.9, the name is chopped up differently to create a similar, but different directory structure, so that same asset ends up in -

AssetFiles/data/00/0C/85/B11C/

Which just after updating an OpenSim 0.8 to 0.8, doesn't exist. In this case the asset does exist in the database, so 0.9 does the usual thing and creates that directory structure to put the file in.

Naturally if the asset was created after the switch to FSassets, it wont exist in the database, and OpenSim 0.9 can't find new assets that should exist. These new assets are in the old 0.8 directory structure.

onefang

onefang

2019-07-17 13:49

administrator   ~0000078

Last edited: 2019-07-17 13:57

My guess about the directory structure is confirmed.

OpenSim/Services/FSAssetService/FSAssetService.cs the HashToPath function, around line 372. The second part of the if statement used to be commented out in 0.8, so it always used the "OSGrid" format. Now we need to set the UseOsgridFormat config option, which is not mentioned anywhere else in the config or source files. It's a boolean that defaults to false, so that the /2/2/2/4/ format is used, instead of the OSGrid /3/3/ format.

Also not documented in release notes, nor the page an FSassets.

onefang

onefang

2019-07-17 14:27

administrator   ~0000079

I think the "best" fix is to try to detect which format is used at run time, then stick with that. If there is none coz it's a fresh grid, or just switched to FSassets, then go with the default, coz we can safely assume OSGrid got it wronger than OpenSim.

Issue History

Date Modified Username Field Change
2019-06-19 04:39 onefang New Issue
2019-06-19 04:39 onefang Status new => assigned
2019-06-19 04:39 onefang Assigned To => onefang
2019-07-17 12:40 onefang Note Added: 0000077
2019-07-17 12:41 onefang Note Edited: 0000077
2019-07-17 12:44 onefang Note Edited: 0000077
2019-07-17 13:49 onefang Status assigned => confirmed
2019-07-17 13:49 onefang Note Added: 0000078
2019-07-17 13:57 onefang Note Edited: 0000078
2019-07-17 14:27 onefang Note Added: 0000079
2019-07-17 19:49 onefang Status confirmed => resolved
2019-07-17 19:49 onefang Resolution open => fixed
2019-07-17 19:49 onefang Fixed in Version => 0.9.0.1 RC0